Random Forests for Hearing Disorder Diagnosis

A new simulation study comparing two statistical methods for diagnosing conditions like misophonia and hyperacusis has found that a machine learning approach may outperform traditional psychometric techniques when hidden biases in assessment questions are present. The research, led by Catherine Bain, Patrick D. Manapat, and Danielle Manapat, directly tested the robustness of Item Response Theory (IRT) and Random Forest (RF) models against a problem known as differential item functioning (DIF).

What Differential Item Functioning Means for Hearing Health

Differential item functioning occurs when a question on a psychological or symptom questionnaire is interpreted or answered differently by distinct groups, even if they have the same underlying level of the trait being measured. For instance, a question about “distress from chewing sounds” might function differently for adolescents versus adults due to social context, not just the severity of misophonia. If DIF is present but not accounted for, it can introduce bias into the diagnostic process, potentially leading to misclassification. Standard psychometric practice often uses IRT models that assume all items are invariant across groups—an assumption this study put to the test.

Simulating Real-World Diagnostic Challenges

The research team used Monte Carlo simulation, a powerful computational method that runs thousands of virtual experiments under controlled conditions. They created simulated datasets representing individuals responding to a diagnostic questionnaire. The simulations systematically varied key factors: the presence and severity of DIF, the sample size, and the characteristics of the assessment scale itself. This allowed them to isolate the impact of DIF on each classification method.

Two primary approaches were compared. The first was a single-group IRT model, which represents current standard practice. It estimates a person’s latent trait (e.g., misophonia severity) from their answers and uses a cut-point for diagnosis. The second was a Random Forest model, a machine learning algorithm that predicts diagnostic class membership directly from the pattern of item responses, without first estimating a latent trait.

Machine Learning Maintains Accuracy Under Bias

The findings were clear. When DIF was absent or minimal, both the IRT-based and RF-based methods produced nearly identical and accurate classification results. The performance gap emerged as the researchers increased the severity of DIF in the simulations.

As DIF severity grew, the classification metrics for the IRT model—such as accuracy and precision—began to decline. The model’s assumption of item invariance was violated, and its diagnostic decisions became less reliable. In contrast, the Random Forest algorithm’s performance remained robust. Its classification accuracy stayed stable across all conditions, showing a notable resistance to the biasing effects of DIF. The authors note this makes RF a viable alternative for diagnostic classification when DIF is suspected but its source, structure, or specific items are unknown, unmeasured, or complex.

Trade-offs in Interpretation and Robustness

The study does not suggest that machine learning is universally superior. Each approach has distinct strengths and limitations relevant for clinicians and researchers. IRT models are highly interpretable; they can precisely show how each question relates to the underlying trait and provide clear diagnostic thresholds. This is valuable for developing and refining assessments, such as those used to identify treatment success factors for misophonia.

Random Forest models, while robust against DIF, are often seen as “black boxes.” It can be difficult to understand exactly why the model made a specific classification decision, which is a significant drawback in clinical settings where explanation is required. The research highlights a practical trade-off: the interpretability of IRT versus the classification robustness of RF when data contains hidden biases.

Implications for Future Assessment and Diagnosis

This research has direct implications for the field of hearing and sound tolerance disorders. Accurate diagnosis is the critical first step toward effective management, whether for hyperacusis, tinnitus, or misophonia. Many diagnostic questionnaires and scales used in research and clinics could contain items with undetected DIF related to age, culture, or comorbid conditions.

The findings suggest that in research contexts where large datasets are available and the primary goal is accurate classification—such as in preliminary screening or for validating new machine learning models for hearing disorders diagnosis—RF methods offer a powerful tool that can withstand certain data biases. For clinical applications, a hybrid approach may be beneficial. RF could be used to flag potential cases with high reliability, while IRT and clinical judgment provide the interpretative framework for final diagnosis and treatment planning.

The study by Bain, Manapat, and Manapat moves the conversation forward by empirically demonstrating how different analytical tools perform under flawed but realistic conditions. It provides a data-driven rationale for selecting assessment methodologies based on the specific challenges present in the data, ultimately aiming for more fair and accurate diagnosis for all patients.

Source: Bain, C., Manapat, P. D., & Manapat, D. (2024). A comparison of item response theory and random forest for diagnostic classification in the presence of differential item functioning. Journal Article. DOI: 10.35566/jbds/bainmmbg.