"Human vs Machine" Validation of a Deep Learning Algorithm for Pediatric Middle Ear Infection Diagnosis

Area of Science:

• Otolaryngology
• Artificial Intelligence in Medicine
• Medical Imaging Analysis

Background:

• Accurate diagnosis of middle ear conditions like acute otitis media (AOM) and otitis media with effusion (OME) is crucial for appropriate treatment.
• Human clinician diagnostic accuracy can vary, potentially leading to misdiagnosis and suboptimal patient outcomes.
• Machine learning algorithms offer a potential tool to enhance diagnostic precision in otology.

Purpose of the Study:

• To compare the diagnostic performance of a developed neural network algorithm against human clinicians in identifying tympanic membrane conditions.
• To evaluate the algorithm's accuracy in differentiating normal tympanic membranes from those with purulent or nonpurulent effusions.
• To assess the potential of machine learning to improve diagnostic accuracy in pediatric otitis media.

Main Methods:

• A retrospective cohort study utilizing a training set of 639 tympanic membrane images.
• Development and training of a neural network algorithm and a commercial image classifier.
• Comparison of model performance against human clinician diagnostic accuracy via a web-based survey on novel image sets.

Main Results:

• The developed neural network achieved a mean prediction accuracy of 80.8%.
• A commercial Google model achieved 85.4% accuracy.
• Human clinicians achieved an average diagnostic accuracy of 65.0% on a validation set, while the developed model achieved 95.5% accuracy on the same data.

Conclusions:

• The developed neural network model outperformed human clinicians in diagnosing tympanic membrane effusions in children.
• Machine learning models show promise in reducing diagnostic errors, potentially decreasing misdiagnoses, unnecessary antibiotic prescriptions, and surgical interventions.
• AI-powered diagnostic tools could significantly impact the management of pediatric ear infections.