Knowledge distillation approach for skin cancer classification on lightweight deep learning model.

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Computational Biology

Background:

• Global incidence of skin cancers is rising, necessitating effective diagnostic tools.
• Deep learning models offer high accuracy for skin cancer detection but are computationally intensive.
• Deploying complex models on resource-constrained devices poses significant challenges.

Purpose of the Study:

• To develop an ultra-lightweight deep learning model for skin cancer detection using knowledge distillation.
• To evaluate the effectiveness of knowledge distillation in transferring knowledge from large to small networks.
• To enable automated skin cancer detection on low-power devices.

Main Methods:

• Implemented knowledge distillation to train lightweight student models.
• Utilized two distinct training strategies: a modified benchmark (Phase 1) and a custom-made model (Phase 2).
• Evaluated performance on two public datasets: HAM10000 and ISIC2019.

Main Results:

• Student models achieved high accuracies: 88.69%–93.24% (HAM10000, Phase 1) and 82.14%–84.13% (ISIC2019, Phase 1).
• Phase 2 accuracies ranged from 88.63%–88.89% (HAM10000) and 81.39%–83.42% (ISIC2019).
• Knowledge distillation significantly improved student model performance, approaching teacher model capabilities.

Conclusions:

• Knowledge distillation is effective for creating accurate, lightweight deep learning models for skin cancer detection.
• The developed models are suitable for deployment on resource-constrained devices.
• This approach facilitates automated skin cancer screening with reduced computational demands.