Related Experiment Video
Updated: Aug 5, 2026
03:14
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Retrieval-Augmented Large Language Model Counseling for Continuous Glucose Monitoring in Diabetes: Source-Masked
Zhijun Guo1, Alvina Lai1, Emmanouil Korakas2
1Institute of Health Informatics, University College London, 222 Euston Road, London, NW1 2DA, United Kingdom, 44 7859995590.
Journal of Medical Internet Research
|August 3, 2026
Summary
A new large language model (LLM) conversational agent (CA) demonstrated superior performance in explaining continuous glucose monitoring (CGM) data compared to clinicians. The LLM-based CA provided more empathetic and actionable responses, enhancing patient understanding for diabetes consultations.
Area of Science:
- Artificial Intelligence in Healthcare
- Digital Health and Diabetes Management
- Clinical Decision Support Systems
Background:
- Continuous glucose monitoring (CGM) is vital for diabetes care, but explaining its patterns is time-consuming for clinicians.
- Existing large language model (LLM) tools for patient-facing CGM interpretation lack robust evaluation against clinician-authored content.
- The study addresses the need for effective LLM support in communicating CGM data to patients.
Purpose of the Study:
- To evaluate a retrieval-grounded LLM-based conversational agent (CA) for its ability to support patient understanding of CGM data.
- To assess if the CA's responses during diabetes counseling are comparable in quality to those written by clinicians.
- To determine the CA's effectiveness in preparing patients for consultations by answering questions about CGM data.
Main Methods:
- Development of a scaffolded LLM-based CA for CGM interpretation and counseling support, designed to provide plain-language explanations without medical advice.
- Construction of 12 CGM-informed cases, including deidentified CGM traces, patient vignettes, and visual materials.
- A multirater evaluation where 6 senior UK diabetes clinicians rated 288 responses (144 CA-generated, 144 clinician-authored) on 6 quality dimensions (accuracy, adherence, actionability, personalization, clarity, empathy).
Main Results:
- CA-generated responses significantly outperformed clinician-authored responses, receiving higher quality scores (4.37 vs. 3.58 on a 5-point scale; P<.001).
- The LLM-CA excelled particularly in empathy (1.062-point difference) and actionability (0.992-point difference).
- Safety flag distributions were similar, and response length did not account for the quality difference.
Conclusions:
- Scaffolded LLM-based systems show potential as adjunct tools for CGM review, patient education, and pre-consultation preparation.
- Findings are based on vignette studies and limited datasets; suitability for autonomous decision-making is not established.
- Prospective validation in clinical workflows is necessary before real-world implementation.