Related Experiment Video
Updated: Jul 9, 2026
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Construction and application of an intelligent tutoring system for resident physicians' medical record writing based
Lin Yang1, Dan Zhao2, Jinyu Zhao3
1Department of Orthopedics, Sichuan Academy of Medical Sciences & Sichuan Provincial People's Hospital, Chengdu, 610072, Sichuan, China.
Objective:
To develop and evaluate a large language model (LLM)-based intelligent tutoring system for medical record writing in residency training and to assess its effectiveness in improving documentation quality and documentation-related learning outcomes.
Methods:
A single-center quasi-experimental controlled before-and-after educational evaluation was conducted from January to December 2025 at a tertiary teaching hospital. Residents (PGY1-PGY3) were allocated to the intervention cohort (LLM-based tutoring system plus conventional teaching) or the control cohort (conventional teaching alone) according to existing teaching arrangements. The system used pseudonymized historical real patient cases to provide structured documentation tasks, automated multidimensional assessment, and personalized feedback. The primary outcome was the overall medical record writing score (0-100). Secondary outcomes included domain-specific documentation scores, documentation error rates, completion time, and resident satisfaction. Statistical analyses included t tests, chi-square tests, effect-size estimation with 95% confidence intervals, and multivariable linear regression.
Results:
A total of 240 residents were included in the analysis (120 per cohort), with comparable baseline characteristics. After 6 months, the intervention cohort demonstrated significantly greater improvement in overall medical record writing scores compared with the control cohort (between-group mean difference in change, 9.0 points; 95% CI, 7.5-10.5; Cohen's d = 1.56; P < 0.001). Improvements were also observed in documentation domains related to history taking, diagnostic analysis, differential diagnosis, and treatment planning. Documentation error rates decreased significantly in the intervention cohort, and residents reported high satisfaction with the system.
Conclusions:
The LLM-based intelligent tutoring system was associated with improved medical record writing quality, documentation efficiency, and resident satisfaction in a residency training setting. These findings support its use as a scalable supplement to conventional documentation training while requiring further validation for broader educational outcomes beyond documentation.