Leveraging FastViT based knowledge distillation with EfficientNet-B0 for diabetic retinopathy severity classification

Area of Science:

• Medical imaging
• Artificial intelligence
• Computer vision

Background:

• Diabetic retinopathy (DR) is a leading cause of blindness globally.
• Accurate and efficient automated DR diagnosis is crucial for early intervention.
• Existing deep learning models often face challenges with computational complexity.

Purpose of the Study:

• To develop a novel deep learning framework, FastEffNet, for enhanced diabetic retinopathy severity classification.
• To reduce computational complexity in DR detection models using knowledge distillation.
• To achieve a balance between diagnostic accuracy and model efficiency.

Main Methods:

• Employed transformer-based knowledge distillation (KD) with FastViT-MA26 (teacher) and EfficientNet-B0 (student).
• Utilized the APTOS blindness detection dataset, including pre-processing, normalization, splitting, and augmentation.
• Evaluated performance using accuracy, precision, recall, F1-score, Kappa scores, MCC, AUC, and computational cost (G FLOPs).
• Incorporated Grad-CAM++ for model interpretability to visualize critical retinal regions.

Main Results:

• EfficientNet-B0, as the student model, achieved the highest accuracy (95.39%) and precision (95.43%).
• The model demonstrated strong performance across various metrics, including F1-score (95.37%), CKS (0.94), WKS (0.97), MCC (0.94), and AUC (0.99).
• Achieved a low computational cost of 0.38 G FLOPs, highlighting its efficiency.

Conclusions:

• FastEffNet, leveraging KD, effectively enhances DR severity classification with a lightweight architecture.
• The optimized EfficientNet-B0 model offers a scalable and cost-effective solution for DR screening.
• Knowledge distillation shows significant potential for developing efficient AI models in medical diagnostics.