A Medical Large Language Model-Based System Improves History-Taking Performance Among Medical Students

To develop a history-taking training and assessment system based on a medical large language model (MedLLM), evaluate its effect on medical students' history-taking competence, and examine its feasibility and effectiveness as a supplement to conventional interview teaching.

An intelligent history-taking training and assessment system was built around a domain-fine-tuned Qwen2.5 large language model. Fifty medical students were assigned in equal numbers to an intervention group and a control group (25 each) using a random number table. The intervention group trained with the system, whereas the control group continued to practice through traditional role-play. Outcomes included system acceptance, practice satisfaction, history-taking examination scores, and agreement between manual and automated scoring.

The intervention group rated the system most favorably for feasibility (4.21±0.95) and interest level (3.76±1.14) (P<0.05), and reported significantly greater satisfaction with flexibility (4.06±0.74) and efficiency (4.10±0.60) than the control group (P<0.001). In the manual examination, the intervention group outperformed the control group (84.31±3.92 vs. 80.56±3.41; P<0.05). Overall agreement between automated and manual scoring was good (intraclass correlation coefficient (ICC)=0.76). It was highest for logical organization (ICC=0.87) and clinical reasoning (ICC=0.85) and moderate for communication skills (ICC=0.63) and humanistic care (ICC=0.65).

As a useful complement to traditional teaching, the MedLLM-based history-taking training and assessment system provides a flexible and efficient practice platform that meaningfully strengthens students' history-taking ability. Its automated scoring agrees well with manual assessment, indicating strong potential for clinical skills education.