Random Forests for Hearing Disorder Diagnosis

Psychometric Tradition vs. Machine Learning: A Diagnostic Face-off

For decades, psychologists and audiologists have relied on validated questionnaires to help diagnose conditions like tinnitus, misophonia, and hyperacusis. The standard method involves a psychometric approach, often using Item Response Theory (IRT). IRT estimates a person’s underlying severity or trait from their questionnaire answers and compares it to a cut-off score for diagnosis. A newer approach uses machine learning algorithms, like Random Forest (RF), which analyze the pattern of item responses to predict diagnostic class directly. Both aim to classify patients accurately, but their performance may differ when a common problem arises: differential item functioning (DIF).

DIF occurs when a questionnaire item behaves differently for different groups of people. For instance, a question about “annoyance from chewing sounds” might be a stronger indicator of misophonia for adolescents than for older adults, even if both groups have the same underlying level of the condition. This hidden bias can distort scores and potentially lead to misclassification. Researchers Catherine Bain, Patrick D. Manapat, and Danielle Manapat set up a simulation study to test how robust IRT and Random Forest are to this problem.

Simulating Real-World Assessment Challenges

The team used Monte Carlo simulations, a computer-based method that generates thousands of synthetic datasets under controlled conditions. They created data representing responses to a hypothetical diagnostic scale. They manipulated key variables: the presence and severity of DIF, the sample size, and the number of items on the scale. They also included a baseline condition using a single-group IRT model, which is common in practice but assumes all items are invariant—that no DIF exists.

The performance of each method—IRT-based classification and Random Forest classification—was judged by standard metrics: accuracy, sensitivity, specificity, and precision. This design allowed them to isolate the effect of DIF on diagnostic outcomes.

DIF Severity Dictates the Winner

The results were clear and depended almost entirely on the level of item bias.

When DIF was absent or very mild, both approaches performed comparably. They achieved similar classification accuracy, showing that machine learning can match traditional psychometrics in ideal, unbiased conditions.

However, as the severity of DIF increased in the simulations, a gap emerged. The classification performance of the IRT-based method declined. Its accuracy, sensitivity, and precision dropped. In contrast, the Random Forest algorithm maintained robust performance. Its classification metrics remained stable across all conditions, even when DIF was strong.

The single-group IRT baseline, which ignores the possibility of DIF, performed worst as bias increased. This highlights a risk in standard practice: applying models that assume item invariance to populations where hidden biases may exist.

Why Random Forest Resists Bias

Random Forest’s resilience likely stems from its fundamental design. It is an ensemble method that builds many decision trees from random subsets of the data and items. When making a final prediction, it aggregates votes from all these trees. This process may naturally dilute the influence of any single biased item. The algorithm does not need to know the source or structure of the DIF to mitigate its effect; the aggregation process provides inherent buffering.

IRT models, especially the simpler ones used for direct classification, are more transparent and interpretable. They provide a clear latent trait score and can pinpoint how each item contributes. But this interpretability comes at a cost. When the model’s core assumption of item invariance is violated by DIF, its estimates—and the classifications derived from them—become less trustworthy.

Practical Implications for Hearing Health Assessment

This simulation study offers practical guidance for clinicians and researchers developing or using assessments for auditory conditions.

In stable, well-understood populations where questionnaires have been rigorously tested for bias, traditional IRT-based classification remains a valid and interpretable choice. For more on the use of questionnaires in understanding these conditions, see our article on Misophonia in Children and Adolescents: Prevalence and Treatment.

When assessing new, diverse, or poorly understood populations where DIF might be present but its details are unknown, Random Forest presents a viable alternative. Its robustness could lead to more reliable screening or diagnostic tools. This aligns with broader trends exploring Machine Learning Advances Hearing Disorder Diagnosis.

The choice involves a trade-off. IRT offers deeper insight into the measurement process itself. Random Forest prioritizes classification stability. The researchers note that hybrid approaches, or using IRT models that explicitly test and account for DIF, could also be solutions. However, detecting and modeling DIF requires knowing which group variables (like age, gender, or culture) might cause bias, and having enough data from those groups. When those factors are unmeasured or the bias is complex, Random Forest’s “black box” robustness may be clinically preferable.

A Tool, Not a Replacement

The work by Bain, Manapat, and Manapat does not suggest abandoning psychometric science. Instead, it highlights that machine learning can supplement it, particularly in challenging assessment environments. For conditions like hyperacusis or misophonia, where patient experiences can vary widely and questionnaires are still evolving, ensuring classification tools are robust to hidden biases is important. As research into Misophonia vs Hyperacusis: Brain fMRI Insights shows, the neural and experiential landscapes of these disorders are complex, which can translate into complexity in how patients respond to questions.

The study, published under DOI 10.35566/jbds/bainmmbg, provides a data-driven argument for considering algorithm robustness alongside model interpretability when building the next generation of diagnostic aids for hearing and sound sensitivity disorders.