Related Experiment Video
Updated: Jul 2, 2026
TBase - an Integrated Electronic Health Record and Research Database for Kidney Transplant Recipients
Published on: April 13, 2021
A temporally Anchored Retrieval-Augmented Generation Framework for Metabolic and Bariatric Surgery Patient Education:
Yash Kumar Atri1, Tom Hartvigsen1, Yung Lee2
1School of Data Science, University of Virginia, Charlottesville, USA.
Background:
Large language models (LLMs) offer promising tools for patient education, yet fixed knowledge cutoffs and hallucination risk limit their clinical utility. Current retrieval-augmented generation (RAG) approaches fail to distinguish between stable clinical knowledge and evolving recommendations.
Methods:
We developed and evaluated bRAGgen, a temporally anchored RAG framework incorporating five modules to enforce clinical protocols for MBS patient education: a semantic knowledge cache, multi-source evidence retrieval with graph-based fusion, uncertainty-aware generation, clinical constraint reranking, and Temporal Fisher Anchoring with Mechanism Selectivity (TFAMS) for adaptive inference. The framework was evaluated using 105 expert-curated free-response questions assessed by a multinational panel of seven specialists (5 surgeons, 2 dietitians) from five countries on a 5-point Likert scale for factuality, clinical relevance, and comprehensiveness. LLM-as-Judge evaluation using ChatGPT-4o provided complementary automated assessment.
Results:
bRAGgen significantly improved response quality across all five base language models tested (p < 0.001), with large effect sizes for higher-capacity models (Cohen's d = 0.96-1.01) and moderate effects for smaller models (Cohen's d = 0.38-0.56) with good inter-rater reliability (Krippendorff's α = 0.72). The largest gains occurred in safety-critical categories including Risks and Complications (+ 1.84 points) and Mental and Emotional Health (+ 1.84 points), suggesting the framework is most impactful where nuanced clinical judgment is essential. LLM-as-Judge evaluation using ChatGPT-4o demonstrated high concordance with expert ratings (Spearman's ρ = 0.94).
Conclusions:
This proof-of-concept study suggests that a multi-module RAG framework with temporal stability anchoring can improve expert-rated LLM response quality for bariatric surgery domain knowledge, though prospective validation in patient-facing settings is needed before clinical implementation.
