Classification of diabetic retinopathy using unlabeled data and knowledge distillation

Area of Science:

• Artificial Intelligence
• Machine Learning
• Medical Imaging

Background:

• Machine Learning (ML) and Artificial Intelligence (AI) show promise in healthcare diagnostics.
• Limited labeled medical data hinders the effectiveness of ML models like Convolutional Neural Networks (CNNs).
• Transfer Learning and Knowledge Distillation offer ways to reuse models but have limitations, especially regarding architectural similarity and complete knowledge transfer.

Purpose of the Study:

• To propose a novel knowledge distillation approach combined with transfer learning.
• To enable the complete knowledge transfer from a large model to a smaller one.
• To leverage unlabeled data for enhanced knowledge transfer in medical image analysis.

Main Methods:

• A new knowledge distillation technique is introduced, integrating transfer learning principles.
• The method utilizes unlabeled data in an unsupervised manner to maximize knowledge transfer.
• The approach aims to distill the complete knowledge from a source model to a target smaller model.

Main Results:

• The proposed method effectively transfers knowledge from complex models to lighter ones.
• Performance of smaller models significantly improved when using unlabeled data and knowledge distillation.
• The approach demonstrated effectiveness in classifying diabetic retinopathy images using public datasets (Messidor, EyePACS).

Conclusions:

• The novel approach successfully addresses the challenge of scarce labeled data in medical imaging.
• It enables efficient knowledge transfer, enhancing the utility of smaller AI models for diagnostics.
• This method holds significant potential for improving AI-driven medical image analysis, particularly for conditions like diabetic retinopathy.