SSD-KD: A self-supervised diverse knowledge distillation method for lightweight skin lesion classification using...

Area of Science:

• Artificial Intelligence in Healthcare
• Medical Image Analysis
• Computational Dermatology

Background:

• Skin cancer diagnosis relies heavily on expert interpretation, posing challenges for large-scale screening.
• Advancements in artificial intelligence (AI) and convolutional neural networks (CNNs) have improved computer-aided diagnosis (CAD) for skin cancer.
• Existing CAD systems often require significant computational resources, limiting their deployment on portable devices.

Purpose of the Study:

• To develop an efficient AI model for skin disease classification adaptable to resource-limited portable devices.
• To propose a novel knowledge distillation (KD) framework, termed SSD-KD, integrating diverse knowledge for enhanced skin disease classification.
• To bridge the gap between high accuracy and computational efficiency in AI-driven skin cancer detection.

Main Methods:

• A novel dual relational knowledge distillation architecture (SSD-KD) was developed, unifying intra-instance relational feature representation.
• The framework integrates existing KD research with a self-supervised training approach and weighted softened outputs.
• Experiments were conducted using the large-scale ISIC 2019 benchmark dataset of dermoscopic images.

Main Results:

• The distilled MobileNetV2 model achieved 85% accuracy in classifying 8 different skin diseases.
• The method demonstrated minimal parameter count and low computing requirements, suitable for portable devices.
• Ablation studies confirmed the effectiveness of the intra- and inter-instance relational knowledge integration strategy.

Conclusions:

• The proposed SSD-KD method offers an effective solution for accurate and efficient skin disease classification on portable devices.
• This study represents the first deep knowledge distillation application for multi-disease classification on a large-scale dermoscopy database.
• The developed approach significantly improves performance compared to state-of-the-art KD techniques.