Integrated image-based deep learning and language models for primary diabetes care.

Area of Science:

• Artificial Intelligence in Healthcare
• Digital Health Solutions
• Ophthalmology and Endocrinology

Background:

• Primary diabetes care and diabetic retinopathy (DR) screening face challenges due to physician shortages, particularly in low-resource settings.
• Effective management requires specialized knowledge often limited in primary care settings.

Purpose of the Study:

• To develop and evaluate an integrated image-language system (DeepDR-LLM) to support primary care physicians (PCPs) in diabetes management and DR screening.
• To improve the quality, empathy, and adherence of diabetes care recommendations.

Main Methods:

• Developed DeepDR-LLM, combining a large language model (LLM) and deep learning for image analysis (DeepDR-Transformer).
• Conducted retrospective evaluations of the LLM module's performance in English and Chinese.
• Performed a prospective, single-center real-world study comparing PCP care with PCP+DeepDR-LLM assisted care.

Main Results:

• The LLM module showed comparable performance to PCPs and residents in English and Chinese.
• DeepDR-Transformer assisted PCPs in DR identification with improved accuracy (92.3% vs. 81.0% unassisted).
• Patients managed with PCP+DeepDR-LLM demonstrated better self-management adherence and higher adherence to DR referrals.

Conclusions:

• DeepDR-LLM effectively supports PCPs in primary diabetes care and DR screening.
• The system enhances patient adherence and the quality of medical recommendations.
• DeepDR-LLM shows significant promise as a digital solution for improving diabetes care outcomes.