Transformer-based classification with enhanced causal explainability from otoscopic images

Otitis media is a major cause of hearing loss, particularly in children. However, nonspecific symptoms and subjective evaluations make its diagnosis challenging. To address this, we developed transformer-based models to classify tympanic membrane conditions from otoscopic images. This approach aims to enhance diagnostic transparency and reliability in clinical settings. We trained vision transformer (ViT) and Data-efficient Image Transformer (DeiT) models on 454 pediatric and adult otoscopic images. These models performed multi-class classification to distinguish between normal, effusion, and tube conditions. For explainability, we utilized Gradient-weighted Class Activation Map (Grad-CAM), Layer-wise Relevance Propagation (LRP), and Attention Rollout (AR). Furthermore, we introduced a hybrid fusion strategy based on Canonical Correlation Analysis. The framework's effectiveness was then evaluated using insertion and deletion causal metrics. The ViT model achieved an accuracy of 97.78% (AUC: 0.998), outperforming DeiT, which reached 93.33% (AUC: 0.994). Notably, ViT attained an F1-score of 97.30% for the effusion class. Among the Explainable Artificial Intelligence (AI) methods, the hybrid LRP and AR approach provided the highest explainability. It yielded an average deletion score of 0.3008 and an insertion score of 0.8918, precisely highlighting critical image features for model predictions. In conclusion, integrating transformer-based models with hybrid explainability methods significantly enhances diagnostic transparency. These advancements foster clinician trust and lay a strong foundation for reliable clinical decision support systems.