DSP-KD: Dual-Stage Progressive Knowledge Distillation for Skin Disease Classification

Area of Science:

• Artificial Intelligence in Medicine
• Medical Image Analysis
• Dermatology

Background:

• Growing global demand for accurate skin disease diagnostics.
• Need for lightweight AI models in medical imaging due to storage and computational constraints.
• Limited research on knowledge distillation for multi-class skin disease classification.

Purpose of the Study:

• To develop an enhanced multi-source knowledge fusion distillation framework (DSP-KD).
• To improve knowledge transfer for AI-assisted dermatoscopic image analysis.
• To balance model accuracy with computational efficiency for practical medical systems.

Main Methods:

• Introduced a dual-stage progressive distillation approach (DSP-KD).
• Maximized mutual information between teacher and student model representations.
• Applied knowledge distillation to ShuffleNetV2 for skin disease classification.

Main Results:

• The distilled ShuffleNetV2 model demonstrated superior performance on ISIC2019 and a private dataset.
• DSP-KD significantly improved knowledge transfer compared to other distillation methods.
• Achieved remarkable effectiveness with a reduced computational burden.

Conclusions:

• DSP-KD offers an effective solution for AI-assisted skin disease diagnosis.
• The framework successfully addresses the challenge of creating lightweight yet accurate diagnostic models.
• Highlights the potential of advanced knowledge distillation techniques in healthcare.