RoS-KD: A Robust Stochastic Knowledge Distillation Approach for Noisy Medical Imaging.

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Machine Learning

Background:

• AI-powered medical imaging offers rapid diagnoses but is hindered by noisy datasets from high annotation costs and human errors.
• Deep learning models struggle with generalization when trained on imperfectly labeled data.

Purpose of the Study:

• To introduce a Robust Stochastic Knowledge Distillation (RoS-KD) framework to improve the performance and robustness of AI models in medical imaging.
• To address the challenge of noisy labels in medical imaging datasets.

Main Methods:

• Developed RoS-KD, a framework that distills knowledge from multiple teacher models trained on overlapping data subsets.
• Trained student models to learn a smooth, well-informed, and robust manifold by mimicking learning from diverse sources.

Main Results:

• RoS-KD demonstrated significant performance improvements on lesion and cardiopulmonary disease classification tasks, achieving >2% and >4% F1-score gains, respectively.
• The framework successfully distilled knowledge from large networks (ResNet-50, DenseNet-121, MobileNet-V2) into smaller student networks (ResNet-18).
• RoS-KD exhibited robustness against adversarial attacks (PGD, FSGM) and outperformed state-of-the-art baselines with ~1% AUC gain in cardiopulmonary disease classification.

Conclusions:

• RoS-KD offers an effective solution for training robust AI models in medical imaging despite noisy datasets.
• The framework enhances diagnostic accuracy and model generalization, paving the way for more reliable AI-assisted healthcare.