Comparing the Generalizability of Multiregional versus Locally Trained Deep Learning Models for Trachoma Detection.

Area of Science:

• Ophthalmology
• Artificial Intelligence
• Public Health

Background:

• Trachoma remains a leading cause of preventable blindness globally.
• Accurate detection of trachomatous inflammation-follicular (TF) is crucial for effective trachoma control.
• Current AI models often lack generalizability due to single-region training data.

Purpose of the Study:

• To compare the performance of AI models trained on single-region versus multiregional datasets for TF detection.
• To evaluate the generalizability of AI models across different geographical regions.
• To assess the advantage of geographically diverse training for AI in ophthalmology.

Main Methods:

• A comparative study using 71,206 everted eyelid photographs from Ethiopia, Niger, and Peru (ages 0-9).
• AI models were trained on single-region and multiregional datasets, then tested across all regions.
• Performance was measured using F1-score, AUROC, and prevalence prediction accuracy.

Main Results:

• Single-region models performed best locally but poorly in other regions.
• Multiregional models demonstrated consistent performance across all test sets (F1=0.85; 0.25; 0.89, AUROC=0.96; 0.79; 0.99).
• Multiregional models accurately predicted TF prevalence globally and showed heatmaps aligning with diagnostic features.

Conclusions:

• Geographically diverse training data is essential for developing generalizable AI models for TF detection.
• Broadly applicable AI models can enhance the scalability and cost-effectiveness of mass drug administration for trachoma.
• AI advancements are key to achieving global trachoma elimination goals.