MedAlmighty: enhancing disease diagnosis with large vision model distillation

Area of Science:

• Medical Imaging and Diagnostics
• Artificial Intelligence in Healthcare
• Machine Learning for Medicine

Background:

• Accurate disease diagnosis is crucial but challenged by limited, heterogeneous medical data.
• Lightweight models lack comprehensive understanding, while large models suffer domain mismatch in specialized medical tasks.

Purpose of the Study:

• To propose MedAlmighty, a knowledge distillation framework bridging general and specialized AI model performance for medical diagnosis.
• To leverage strengths of large vision models and lightweight CNNs to overcome data limitations in medical AI.

Main Methods:

• Utilized DINOv2 (large vision model) as a frozen teacher and a lightweight CNN as a trainable student.
• Employed knowledge distillation with hard labels and soft targets from the teacher.
• Adopted a hybrid loss function combining cross-entropy and Kullback-Leibler divergence for efficient, domain-aware learning.

Main Results:

• MedAlmighty significantly improved disease diagnosis performance on sparse and diverse medical datasets.
• The model outperformed baselines by integrating generalizable representations with specialized knowledge.
• Demonstrated enhanced robustness and accuracy in complex diagnostic scenarios.

Conclusions:

• MedAlmighty effectively enhances lightweight medical models using general-domain representations from frozen large vision models via distillation.
• This framework offers a solution for data scarcity and domain gap issues in medical imaging.
• Future work includes extending distillation to multimodal medical data.