A dual-teacher cross-modal knowledge distillation framework integrating visual and textual modalities for neonatal...

Neonatal jaundice remains a major health challenge, particularly in resource-limited environments where reliable diagnosis is not readily accessible. While image-based and AI-driven approaches have demonstrated encouraging diagnostic performance, their limited generalizability across different datasets, populations, and clinical settings remains a major challenge.

This study proposes a Dual-Teacher Cross-Modal Knowledge Distillation (DT-CMKD) framework for efficient neonatal jaundice diagnosis. Complementary knowledge is distilled from two teachers: a vision-based MambaVision model capturing rich spatial features from neonatal images and a text-based model combining multimodal LLaMA-3-17B and CLIP ViT-B/32 text encoder to generate semantic embeddings from clinically informed neonatal image descriptions. The distilled knowledge from both teachers is transferred to a lightweight ResNet-50 student via multi-level distillation integrating logit-based supervision, feature alignment, and cross-entropy optimization.

The proposed model was trained and validated on the Normal and Jaundiced Newborn (NJN) dataset. Further, we introduce JaundiSet-NG and NeoJaundice as two independent external validation datasets representing distinct demographic populations and image acquisition protocols to assess generalizability. The proposed framework outperformed conventional unimodal baseline models, achieving an improved 97.37% accuracy and 98.25% F1-score. Model explainability was validated through Grad-CAM visualization and LLM-generated expert-like text descriptions.

This study presented a knowledge distillation-based multimodal framework that enhances non-invasive neonatal jaundice diagnosis while significantly reducing inference cost, demonstrating strong potential for deployment in resource-constrained clinical environments.