Semantic-Oriented Visual Prompt Learning for Diabetic Retinopathy Grading on Fundus Images

Area of Science:

• Ophthalmology
• Medical Imaging
• Artificial Intelligence

Background:

• Diabetic retinopathy (DR) grading is crucial for patient care but faces challenges in model development due to data and resource demands.
• Existing methods often require extensive fine-tuning of large-scale pre-trained models (LPMs), demanding significant computational resources and high-quality datasets.

Purpose of the Study:

• To investigate the knowledge transferability of LPMs for efficient DR grading using prompt learning.
• To develop a novel prompt learning method, Semantic-oriented Visual Prompt Learning (SVPL), to enhance semantic perception and task-specific knowledge extraction from LPMs for DR grading.

Main Methods:

• Proposed Semantic-oriented Visual Prompt Learning (SVPL) which utilizes learnable prompts for each DR level without additional annotations.
• Introduced a Contrastive Group Alignment (CGA) module to align prompt groups within a task-specific semantic space.
• Developed a Hierarchical Semantic Delivery (HSD) adapter module to facilitate efficient knowledge mining and model convergence.

Main Results:

• SVPL achieved competitive results compared to full-tuning methods while requiring fewer learnable parameters.
• Experiments on three public DR grading datasets demonstrated SVPL's superior performance over existing transfer tuning and DR grading methods.
• Analysis confirmed the advantage of generalized knowledge from LPMs for DR grading on fundus images.

Conclusions:

• SVPL offers an efficient and effective approach for developing DR grading models by leveraging LPMs through prompt learning.
• The proposed method enhances semantic understanding and knowledge extraction, paving the way for more accessible and accurate DR diagnosis.