External Validation of a Custom-Trained Artificial Intelligence Model for Canine Ear Canal Disease and Comparison...

Otitis externa (OE) is common in dogs, yet accurate otoscopic diagnosis remains challenging. Artificial intelligence (AI) may augment clinical decision-making in otoscopy. Previous work suggested that dataset size and duplicate images affect model performance.

External validation of four YOLOv8n models using new images and comparison with six veterinary professionals of varying experience. We expected that models trained on larger, deduplicated datasets would demonstrate superior performance, and that the best-performing model would outperform non-specialist veterinary professionals while remaining comparable to board-certified referral clinicians.

Four YOLOv8n models (A-D) were trained using different dataset compositions and externally validated on 300 previously unseen otoscopic images (100 healthy, 100 OE, 100 ear canal masses). Predictions were compared with reference labels assigned by a board-certified referral clinician. Six veterinary professionals independently classified the same images, blinded to ground truth and AI outputs. Performance was assessed across multiple confidence thresholds (0.25-0.75) using overall agreement, Cohen's κ, sensitivity and specificity.

Model D (largest deduplicated dataset) demonstrated the highest external performance, achieving 66.3% agreement and κ = 0.508 at a 0.25 confidence threshold. Classwise sensitivity/specificity for Model D (0.25) were: healthy (90.0%/69.0%), OE (59.0%/75.5%) and mass (50.0%/93.5%). Veterinary professionals outperformed all AI models (median agreement 77.2%, κ = 0.658), with the referral clinician achieving the highest concordance.

Model D generalised best during external validation yet remained inferior to veterinary professionals. These results support the model's role as a decision-support tool rather than a replacement for clinical expertise.