Random Forest for Hearing Disorder Diagnosis

A new simulation study directly compares two statistical methods for turning questionnaire answers into a diagnosis, with a particular focus on how well they perform when the questions themselves behave differently across subgroups of patients. The research, led by Catherine Bain and colleagues, found that while traditional psychometric methods and modern machine learning perform equally well under ideal conditions, a common data problem called differential item functioning (DIF) can significantly degrade the accuracy of the traditional approach, while the machine learning method remains stable.

How the Study Compared Diagnostic Approaches

Bain, Manapat, and Manapat used Monte Carlo simulations to create thousands of virtual datasets. This allowed them to precisely control conditions and test how each classification method reacted. They simulated responses to a hypothetical questionnaire designed to measure a latent psychological trait, like sound sensitivity or distress.

The core conditions they varied were the presence and severity of Differential Item Functioning (DIF). DIF occurs when individuals from different groups (e.g., different ages, genders, or cultural backgrounds) who have the same level of the underlying trait have different probabilities of endorsing a specific questionnaire item. For instance, a question about “anger in response to chewing sounds” might function differently for teenagers versus older adults, even if they have the same overall level of misophonia. DIF is a known challenge in assessing conditions like misophonia and hyperacusis, where subjective experiences and reporting can vary.

The team compared two classification pipelines. The first was a single-group Item Response Theory (IRT) model, representing standard psychometric practice. IRT estimates a person’s latent trait score from their item responses and then uses a cut-point on that score to assign a diagnosis. Critically, the single-group model assumes all items are invariant (no DIF) across populations. The second approach used a Random Forest (RF) machine learning algorithm, which learns complex patterns directly from the raw item responses to predict diagnostic class membership, without first estimating a latent trait score.

Machine Learning Showed Resilience to Problematic Data

When the simulated data contained no DIF, both the IRT and Random Forest methods produced diagnostic classifications with comparable accuracy, sensitivity, and specificity. This confirms that both are valid approaches under ideal measurement conditions.

The results diverged sharply as DIF was introduced and its severity increased. The classification performance of the IRT-based method systematically declined. Because the single-group IRT model incorrectly assumed all items functioned the same for everyone, its latent trait estimates became biased, leading to less accurate diagnostic decisions at the cut-point.

In contrast, the Random Forest algorithm’s performance remained robust across all levels of DIF severity. The machine learning model, by learning directly from the response patterns, was apparently able to adapt to or accommodate the DIF without a significant loss in its ability to correctly classify individuals. This suggests RF has a practical advantage in real-world settings where researchers or clinicians may suspect DIF exists—due to factors like age or co-occurring neurological conditions—but cannot easily measure or model its complex structure.

Trade-offs for Clinical and Research Applications

The authors clarify that their findings do not make one method universally superior. Each has strengths and limitations that suit different purposes.

The IRT framework is highly interpretable. It provides a clear, continuous latent trait score (e.g., a “hyperacusis distress” score) that indicates severity, and it can identify which specific items are most informative at different levels of the trait. This is valuable for tracking individual progress over time or for developing concise assessments. However, its reliance on strict statistical assumptions is a weakness when those assumptions, like item invariance, are violated.

Random Forest, as used in this study, is primarily a classification engine. It excels at predicting the diagnostic category (e.g., “meets criteria” or “does not meet criteria”) with stability, even in messy data. Its “black box” nature is a drawback; it is less clear how it arrived at a decision or how to derive a fine-grained severity score from its output. This aligns with broader trends in using machine learning for hearing disorder diagnosis, where the focus is often on pattern recognition.

Choosing the Right Tool for the Task

For clinicians and researchers, the choice may depend on the assessment goal. If the primary need is a reliable, consistent diagnostic decision in a diverse population where DIF is a potential concern, a well-trained machine learning model like Random Forest could be a viable and robust alternative.

If the goal is to understand a patient’s specific symptom profile, measure subtle changes in severity after an intervention like TMJ therapy for tinnitus, or refine a theoretical model of a disorder, the interpretable scores from an IRT approach—provided DIF is carefully checked and managed—remain highly valuable.

The work by Bain and colleagues, available via DOI 10.35566/jbds/bainmmbg, provides empirical evidence for a strategic choice in assessment methodology. It underscores that in the complex landscape of auditory and psychological conditions, the statistical tool used to define the diagnosis itself can meaningfully influence results, especially when the patient population is not uniform.