Machine Learning Models for Hearing Disorders Diagnosis

Accurately diagnosing conditions like tinnitus, hyperacusis, and misophonia often relies on patient questionnaires. These psychological assessments ask a series of questions to gauge symptom severity and determine if a person meets diagnostic criteria. The statistical methods used to score these tests are foundational to getting the diagnosis right. New simulation research by Catherine Bain, Patrick D. Manapat, and Danielle Manapat directly compares two powerful approaches, finding one maintains its accuracy better when hidden biases creep into the questions.

The Diagnostic Challenge: Latent Traits and Biased Questions

Clinicians and researchers can’t directly measure the distress caused by tinnitus or the emotional reactivity in misophonia. These are latent traits inferred from how patients answer a set of items on a questionnaire. For decades, the gold standard for modeling these responses has been Item Response Theory (IRT). IRT estimates a person’s latent trait score and then compares it to a cut-point for classification. A core assumption of standard IRT is measurement invariance: a question about “annoyance” or “avoidance” should have the same difficulty and discrimination for all groups, regardless of age, gender, or comorbid conditions.

This assumption is often violated by Differential Item Functioning (DIF). DIF occurs when a question performs differently for different groups, even among people with the same underlying level of the trait. For example, a question about “difficulty in work meetings” might function differently for retired individuals versus working professionals with similar tinnitus severity. DIF introduces a hidden bias that can distort scores and misclassify patients. Single-group IRT models, commonly used in practice, typically ignore DIF.

Machine learning offers a different path. Algorithms like Random Forest (RF) bypass latent trait estimation. They learn complex patterns directly from the raw item responses to predict diagnostic class membership. The question Bain and colleagues asked was which method holds up better when DIF is present in the data.

Simulating Real-World Diagnostic Scenarios

The research team used Monte Carlo simulation, a method that generates many synthetic datasets with known properties, to test the methods under controlled conditions. They created data mimicking realistic psychological scales, varying key factors: sample size, test length, the correlation between items, and—critically—the presence and severity of DIF. They introduced DIF that was either balanced (affecting different groups but canceling out in total test scores) or unbalanced (leading to systematic score distortion). They then compared how well the IRT-based classification and the Random Forest classifier recovered the true diagnostic status of each simulated person.

Machine Learning Maintains Performance as Bias Increases

When DIF was absent or minimal, the results were a draw. Both IRT and Random Forest produced comparable and accurate classification metrics, including sensitivity, specificity, and overall accuracy. This confirms both are valid approaches under ideal measurement conditions.

The divergence appeared as DIF severity increased. The classification performance of the single-group IRT model consistently declined. The bias in the items led to errors in the latent trait estimates, which in turn led to more misclassification. In contrast, the Random Forest algorithm’s performance remained remarkably stable and robust across all levels of DIF severity. It was able to learn from the complex response patterns, including those influenced by DIF, without its predictive accuracy suffering.

“These findings suggest that RF may maintain more stable classification performance than IRT-based classification when DIF is present but not explicitly accounted for in the model,” the authors conclude. This makes RF a strong alternative for diagnostic classification when DIF is suspected but its specific source or pattern is unknown, unmeasured, or too complex to easily model with traditional techniques.

Implications for Hearing Health and Sound Sensitivity Assessment

This research has direct implications for the field of auditory disorders. Questionnaires are central to diagnosing and measuring the impact of misophonia and hyperacusis, as well as for evaluating outcomes in tinnitus retraining therapy or tinnitus management counseling. Patient populations are diverse, and DIF can arise from cultural, linguistic, age-related, or disorder-specific factors. If a standard tinnitus handicap inventory contains DIF related to occupational status, for instance, it could lead to systematic over- or under-diagnosis in certain groups.

The study highlights a fundamental trade-off. IRT provides deep interpretability; researchers can pinpoint which items are difficult and how they relate to the latent trait. This is valuable for refining scales. Random Forest, while excellent at stable prediction, operates more as a “black box,” making it harder to understand why it made a specific classification decision. For pure diagnostic classification where robustness is the priority—especially in initial screening—machine learning offers a compelling advantage. For research aimed at understanding the precise structure of a condition, psychometric methods remain essential.

The work by Bain, Manapat, and Manapat does not discard one method for the other. Instead, it provides clear evidence for when each tool is most effective. As assessment in hearing health advances, incorporating methodological checks for DIF and considering robust machine learning approaches could lead to fairer, more accurate diagnoses for patients experiencing tinnitus, misophonia, and hyperacusis.

Source: Bain, C., Manapat, P. D., & Manapat, D. (2024). A comparison of item response theory and random forest for diagnostic classification in the presence of differential item functioning. Journal of Behavioral Data Science, 10.35566/jbds/bainmmbg.