Semi-supervised Learning with Online Knowledge Distillation for Skin Lesion Classification.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Deep learning methods for skin lesion analysis typically require large amounts of labeled data, which are costly and time-consuming to acquire.
• Existing supervised approaches face challenges in annotation burden and resource constraints.

Purpose of the Study:

• To develop a novel semi-supervised deep learning approach for skin lesion classification that minimizes the reliance on labeled data.
• To enhance the performance and label efficiency of skin lesion classification models.

Main Methods:

• A semi-supervised deep learning framework integrating ensemble learning with online knowledge distillation.
• Training an ensemble of convolutional neural network (CNN) models and using knowledge distillation to transfer insights among ensemble members.
• Enabling deployment of individual models with comparable performance to the ensemble, suitable for resource-constrained settings.

Main Results:

• The proposed knowledge-distilled individual models outperform independently trained models.
• The approach achieves state-of-the-art performance on ISIC 2018, 2019, and 2020 datasets.
• Demonstrates significant label efficiency, achieving comparable performance to fully supervised baselines with substantially less labeled data (e.g., 10% vs. 20% on ISIC 2018).

Conclusions:

• The novel semi-supervised approach effectively reduces the annotation burden in skin lesion classification.
• The method offers superior label efficiency and practical relevance for real-world applications.
• The developed technique achieves high performance and efficiency, making it suitable for resource-limited environments.