Related Experiment Video
Updated: Jun 21, 2026
Project-Based Learning Guidelines for Health Sciences Students: An Analysis with Data Mining and Qualitative Techniques
Published on: December 9, 2022
Voluntary use of LLM-powered virtual standardized patients and medical students' interview performance: An
Xiaoxing Gao1, Jun Feng2, Lian Duan3
1Department of Respiratory and Critical Care Medicine, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China.
Background:
Large language model (LLM)-powered Virtual Standardized Patients (VSPs) offer scalable practice opportunities for clinical interviewing, but their added value within established human Standardized Patient (SP) curricula remains unclear. This study examined how self-directed VSP engagement relates to interview performance on traditional SP assessments through the lens of self-regulated learning theory.
Methods:
We analyzed VSP usage logs and SP assessment data from fourth-year medical students (n = 92) enrolled in a 7-week diagnostics course with weekly human SP sessions and optional VSP access. Engagement was measured by valid sessions and total question-answer (QA) pairs. The primary outcome was the composite SP interview score (0-100). Associations were evaluated using correlation and regression analyses.
Results:
Students generated 359 valid dialogues comprising 19,380 QA pairs (hallucination rate 0.34%). Median engagement was 2 sessions (IQR 1-7) and 132 QA pairs (IQR 38-323). Session frequency correlated modestly with SP scores (ρ = 0.25, p=.016); each additional session predicted a 0.15-point increase (R2=0.070). Students with ≥7 sessions outperformed those with fewer (94.1 ± 2.0 vs 92.5 ± 2.4, p=.013). QA volume showed a stronger association (ρ = 0.28, p=.006), explaining 10.6% of variance. The high-QA group (≥132) scored higher than the low-QA group (93.3 ± 2.3 vs 92.3 ± 2.3, p=.030).
Conclusions:
Voluntary VSP use within a human SP curriculum was associated with small but measurable differences in interview performance. Findings align with self-regulated learning and deliberate practice frameworks-students who independently sought repeated, feedback-rich practice tended to achieve higher scores. These observational data do not establish causality. VSPs may serve as scalable cognitive scaffolds for self-directed skill refinement, though their benefits likely vary with learner motivation and baseline ability.