DeepFusion-HSK: a multimodal deep learning for subtype classification of herpes simplex keratitis with multicenter validation
Related Experiment Videos
Zuhui Zhang1, Tao Lv1, Jiahao Tang1
1National Clinical Research Center for Ocular Diseases, Eye Hospital, Cixi Biomedical Research Institute, Wenzhou Medical University, Wenzhou, China.
Introduction:
To develop and evaluate a multimodal deep learning (DL) model for subtype classification of viral keratitis.
Methods:
This retrospective multicenter study included 3,501 slit-lamp images (807 eyes) as the internal development set and two external validation cohorts (2,957 images total). DeepFusion-HSK integrates three complementary illumination modalities-diffuse light imaging (DLI), slit-beam imaging (SBI), and fluorescein staining imaging (FSI)-through a novel fusion architecture. The framework uses a tri-way higher-order interaction layer to capture cross-modal pathological signatures, dual-task learning with auxiliary heads to prevent modality-specific feature degradation, and a dynamic gating mechanism that uses high-confidence coarse priors to calibrate fine-grained predictions and mitigate softmax probability spillover. To simulate real-world data deficiency, we evaluated model resilience under complete FSI dropout.
Results:
On the internal test set, DeepFusion-HSK achieved an accuracy of 85.1% (95% CI: 78.5%-90.0%) and AUCs of 0.947-0.949 for discriminating epithelial herpes simplex keratitis (E-HSK) and non-epithelial herpes simplex keratitis (NE-HSK), significantly outperforming Swin-Transformer, EfficientNetV2, and ResNet50. External validation confirmed robust generalization, with accuracies of 81.7% on Dataset I and 78.2% on Dataset II, demonstrating clinical resilience. Ablation studies confirmed the hierarchical synergy of multimodal inputs: the best single modality (FSI) yielded 78.5% accuracy, the optimal dual-modality combination (DLI + FSI) reached 81.7%, and full tri-modal fusion improved performance to 85.1%, validating that integrated global morphology, depth contour, and superficial ulcer signatures are essential for robust HSK subtyping.
Discussion:
DeepFusion-HSK offers a robust, generalizable, and clinically deployable solution for therapeutic HSK subtyping. Its resilience to missing imaging modalities makes it well suited for heterogeneous real-world ophthalmic practice, providing a decision-support tool to reduce vision-threatening misdiagnoses and guide therapy.
