Multi-stage knowledge distillation with layer fusion-based deep learning approach for skin cancer classification

Area of Science:

• Dermatology
• Artificial Intelligence
• Medical Imaging

Background:

• Skin cancer is a prevalent global health issue, necessitating advancements in diagnostic tools.
• Early detection and accurate classification of skin lesions are crucial for effective treatment and improved patient outcomes.

Purpose of the Study:

• To develop and evaluate a novel multi-stage knowledge distillation deep learning technique for classifying skin lesion cells.
• To enhance the accuracy, efficiency, and interpretability of skin cancer diagnostic models.

Main Methods:

• Utilized the HAM10000 dataset with augmentation and preprocessing for robust model training.
• Implemented a multi-stage knowledge distillation approach with a hybrid teacher model (ViT and ConvNeXT) and a distilled model (CNN and EfficientNet).
• Incorporated layer fusion, intermediate exits, post-training quantization, and explainable AI (XAI) techniques (Grad-CAM, Score-CAM, LIME).

Main Results:

• The optimized distilled model achieved high performance metrics: 95.88% accuracy, 95.91% F1-score, and 99.02% AUC.
• Demonstrated significantly reduced training and inference times (61 ms/step and 15 ms/step, respectively).
• Enhanced model interpretability through the application of multiple XAI techniques.

Conclusions:

• The proposed multi-stage knowledge distillation with layer fusion offers a significant improvement in skin lesion classification accuracy and efficiency.
• The optimized and quantized model provides a computationally efficient and interpretable solution for skin cancer diagnosis.
• The study provides accessible code and a quantized model for further research and application in skin cancer detection.