Stepwise self-knowledge distillation for skin lesion image classification.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Medical Imaging

Background:

• Self-knowledge distillation is popular for medical image classification, using identical teacher and student models.
• Current methods struggle with defining effective learning objectives for student models, limiting performance gains.
• Dermatological image classification requires robust models to accurately diagnose skin conditions.

Purpose of the Study:

• To introduce a novel Stepwise Self-Knowledge Distillation (SW-SKD) framework to enhance dermatological image classification.
• To address limitations in current self-knowledge distillation by improving the determination of learning objectives.
• To boost the performance of student models in classifying dermatological images.

Main Methods:

• Developed the SW-SKD framework with a stepwise distillation strategy.
• Incorporated a feature rectification block (FRB) using attention-corrected features as learning objectives.
• Implemented a logit rectification block (LRB) for logit-based knowledge distillation, adjusting predictions to match correct indices.

Main Results:

• SW-SKD significantly improved dermatological image classification performance on HAM10000, ISIC2019, and Dermnet datasets.
• On HAM10000, Precision improved by 0.8%-1.4% and Recall by 0.9%-2.1% compared to the second-best method.
• On ISIC2019, average Precision improved by 0.5%-0.9% and average Recall by 0.7%-1.1%.

Conclusions:

• The proposed SW-SKD framework effectively enhances student model performance in dermatological image classification.
• Stepwise distillation using FRB and LRB provides superior learning objectives compared to existing methods.
• SW-SKD demonstrates significant potential for improving diagnostic accuracy in medical image analysis.