Feasibility of Large Language Model-Based Standardized Virtual Patients to Support Clinical Decision-Making Training...

Clinical decision-making training in operative dentistry commonly relies on real or standardized patients to develop undergraduate students' ability to deliver safe, effective, and patient-centered care. However, training with real or standardized patients can be limited in scalability, cost-effectiveness, and accessibility. Large language models, with their human-like language capabilities, may have the potential to simulate patients in clinical encounters and help overcome some limitations associated with traditional training approaches.

This study aimed to evaluate the feasibility of using large language model-based standardized virtual patients to support undergraduate dental students' clinical decision-making training in operative dentistry.

This mixed methods cross-sectional feasibility study was conducted during a simulation-based clinical decision-making training session in the Operative Dentistry and Cariology course at the College of Dentistry, King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia. Eligible participants were second-year undergraduate dental students enrolled in the course. A convenience sampling approach was used, with all eligible students (N=50) invited to participate. A total of 41 students completed the study, 23 (56%) of whom were male. The students were divided into 8 groups. Each group interacted with 2 standardized virtual patients powered by ChatGPT-4o (OpenAI) through the Chatbase platform to complete comprehensive history-taking and then reviewed the standardized virtual patients' intraoral photographs and bitewing radiographs. For each standardized virtual patient, students as a group recorded diagnoses, performed a risk assessment, and formulated a treatment plan. Students then completed the Student Satisfaction and Self-Confidence in Learning questionnaire. The quality of the standardized virtual patient responses and overall dialogue realism were evaluated using the Dialogue Authenticity Scale. The dialogues were also thematically analyzed to identify authenticity-undermining response features and explore their context and underlying causes.

Students perceived the simulation-based training session positively, with all questionnaire items showing high median scores (4.00-5.00 on a 5-point scale), and both item-level IQRs and 95% CIs spanning no more than 1.0 scale point. In addition, standardized virtual patient responses were largely authentic, with an overall median authenticity rating of 4.50 (IQR 4.00-5.00; 95% CI 4.00-5.00) on a 6-point scale across all interactions. However, several authenticity-undermining response features were identified, including responses that were inconsistent with typical human behavior, contained information beyond a patient's likely knowledge, or were factually incorrect.

This proof-of-concept study supports the feasibility of implementing large language model-based standardized virtual patients in undergraduate simulation-based clinical decision-making training in operative dentistry. In a dental context where this application has been only minimally evaluated, this study provides early evidence of positive student perceptions, acceptability, and largely authentic dialogue, while also identifying important performance limitations. Further research is warranted to optimize performance and to evaluate the educational effectiveness of this approach in improving undergraduate students' clinical skills and knowledge.