Random Forests for Hearing Disorder Diagnosis

Comparing Statistical Models for Hearing Disorder Diagnosis

Diagnosing conditions like misophonia or hyperacusis often relies on patient questionnaires. Clinicians use statistical models to interpret these responses and determine if someone meets diagnostic criteria. A new simulation study by researchers Catherine Bain, Patrick D. Manapat, and Danielle Manapat tested two powerful methods: a traditional psychometric model called Item Response Theory (IRT) and a machine learning model known as Random Forest (RF). Their work, published in the Journal of Behavioral Data Science, reveals a critical difference in how these models handle biased test questions, with direct implications for accurate diagnosis in hearing health.

How the Study Tested Diagnostic Models

The research team used Monte Carlo simulations, a method that runs thousands of computer-generated experiments. They created virtual populations and psychological scales, mimicking real-world diagnostic assessments. The key variable they manipulated was Differential Item Functioning (DIF). DIF occurs when a question on a questionnaire is inherently biased—it might be easier for one demographic group to endorse than another, even if both individuals have the same underlying level of the condition. For example, a question about “anger towards chewing sounds” could be interpreted differently across cultures.

They compared two classification approaches. The first was a standard, single-group IRT model, which estimates a person’s latent trait (e.g., misophonia severity) and uses a cut-point for diagnosis. This model assumes all test items work identically for everyone, ignoring DIF. The second was a Random Forest algorithm, which uses an ensemble of decision trees to predict diagnostic class directly from the pattern of item responses, without first estimating a latent trait.

Machine Learning Maintains Accuracy When Bias Is Present

When the simulated data contained no DIF, both the IRT and RF models performed with comparable and high accuracy. Classification metrics like sensitivity and specificity were nearly identical. This confirms that both are valid tools under ideal, invariant conditions.

The results diverged sharply as the researchers introduced and increased the severity of DIF in the test items. The classification performance of the single-group IRT model “declined,” the authors report. Because IRT did not account for the group-specific bias in the items, its latent trait estimates became less accurate, leading to more diagnostic errors. In contrast, the Random Forest model “maintained robust performance across conditions.” Its classification accuracy remained stable even as DIF severity increased.

This stability suggests RF can handle complex, unmeasured biases in questionnaire data more effectively than a standard IRT approach. For clinicians and researchers, this is significant. In practice, the source of DIF—whether from age, gender, cultural background, or comorbid conditions—is often unknown or too complex to model explicitly.

Implications for Hearing Disorder Assessment

This finding has practical weight for the fields of tinnitus, hyperacusis, and misophonia. Diagnostic criteria for these conditions are often operationalized through self-report scales like the Misophonia Questionnaire or the Hyperacusis Questionnaire. If these scales contain items with DIF, reliance on traditional psychometric models could lead to systematic misclassification for certain patient groups. Someone might be under- or over-diagnosed based on demographic factors unrelated to their actual symptoms.

The study proposes RF as a “viable alternative for diagnostic classification when DIF is suspected but its source or structure is unknown, unmeasured, or complex.” This is particularly relevant given the distinct neurological profiles of these disorders and the need for precise diagnostic tools to separate them. Accurate classification is the first step toward appropriate management, whether that involves sound therapy, cognitive behavioral approaches, or other interventions discussed in our resource on misophonia treatment in young people.

The Trade-Off: Interpretability vs. Robustness

The authors note that the choice between IRT and RF involves a trade-off. IRT is highly interpretable; it provides a clear score on a latent trait, showing not just *if* someone has a condition, but *how much* of the trait they possess. This is valuable for tracking symptom progression or treatment response. RF, as a “black box” machine learning model, excels at prediction but offers less direct insight into the continuum of severity. It tells you the “what” (the diagnosis) more reliably under biased conditions but is less clear on the “how much.”

This research aligns with a broader movement exploring machine learning in hearing disorder diagnosis. The study by Bain and colleagues provides specific, empirical support for using these algorithms to improve diagnostic consistency, especially in diverse populations.

A New Consideration for Clinicians and Researchers

The work of Bain, Manapat, and Manapat does not discard traditional psychometrics. Instead, it highlights a specific vulnerability—unac (NAC supplement)counted-for DIF—and offers a data-driven solution. For large-scale screening or epidemiological work where demographic diversity is high and DIF is a risk, RF may provide more dependable classification. For clinical settings where understanding a patient’s specific position on a severity spectrum is needed, IRT remains essential, though analysts should actively test for and control DIF.

As diagnostic questionnaires for hearing-related disorders continue to be developed and validated, this study makes a strong case for comparing multiple modeling approaches. The goal is to ensure that a diagnosis reflects true symptomatology, free from the hidden bias of the questions themselves.

Source: Bain, C., Manapat, P. D., & Manapat, D. (2024). A Comparison of IRT- and Random Forest-Based Diagnostic Classification in the Presence of Differential Item Functioning. Journal of Behavioral Data Science. DOI: 10.35566/jbds/bainmmbg