Efficient image classification through collaborative knowledge distillation: A novel AlexNet modification approach.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Lightweight models are crucial for image classification in resource-limited environments.
• Traditional knowledge distillation methods may not fully leverage teacher model information.
• AlexNet architecture requires optimization for efficiency.

Purpose of the Study:

• To introduce an innovative, lightweight image classification technique.
• To enhance knowledge transfer using a novel distillation method.
• To maintain high accuracy and robustness in computationally constrained settings.

Main Methods:

• Modified AlexNet architecture incorporating depthwise-separable convolution layers.
• Teacher-Student Collaborative Knowledge Distillation (TSKD) for dual-layered learning (intermediate and final layers).
• Development of specialized loss functions to balance complexity and efficiency.

Main Results:

• The lightweight model achieves high accuracy and robustness in image classification tasks.
• TSKD facilitates more efficient knowledge transfer compared to conventional methods.
• Architectural optimizations and specialized loss functions improve computational efficiency.

Conclusions:

• The proposed lightweight model with TSKD is effective for image classification under computational constraints.
• Depthwise-separable convolutions and TSKD are key innovations for efficient knowledge transfer.
• This approach offers a valuable solution for deploying advanced image classification in limited-resource scenarios.