Joint learning method with teacher-student knowledge distillation for on-device breast cancer image classification

Area of Science:

• Medical Imaging
• Computer Science
• Artificial Intelligence

Background:

• Deep learning models like AlexNet, VGG, and ResNet show promise in classifying breast cancer histopathological images.
• However, their high computational complexity and large number of parameters limit their use on resource-constrained devices.

Purpose of the Study:

• To develop a lightweight deep learning model for breast cancer histopathological image classification using knowledge distillation.
• To address the limitations of existing complex models for practical deployment.

Main Methods:

• Employed knowledge distillation with VGG and ResNeXt teacher models to train smaller, efficient student models.
• Utilized an adaptive joint learning approach to transfer knowledge from teacher model outputs and feature maps to student models.

Main Results:

• The ResNeXt-based student model achieved a 97.09% recognition rate on the BreakHis dataset.
• This lightweight model demonstrated a significant reduction in parameters, GPU memory usage, and a high compression rate compared to its teacher model.
• The performance drop was minimal (1.75%) compared to the teacher model, indicating acceptable outputs.

Conclusions:

• The proposed lightweight knowledge distillation model offers a practical and efficient solution for breast cancer histopathological image classification.
• This approach balances high accuracy with reduced computational demands, making it suitable for deployment on devices with limited resources.