Leveraging artificial intelligence to accelerate competency development in diagnostic reasoning of medical students

Diagnostic errors remain a critical patient safety concern, yet teaching diagnostic reasoning is challenging due to its complex, context-specific nature. Traditional teaching methods emphasize analytical reasoning but often neglect non-analytical, pattern recognition processes that are grounded in script theory. Script theory highlights illness scripts-structured mental representations of diseases-as foundational to diagnostic expertise. Advances in artificial intelligence (AI) offer opportunities to support illness script formulation through personalized, scalable feedback. This study evaluated whether chatbot-enabled instruction and feedback improve medical students' illness script quality compared with conventional instruction techniques.

We conducted a parallel-group randomized controlled trial with 83 medical students at Prince of Songkla University, Hat Yai, Thailand. Participants were randomized to conventional instruction or chatbot-enabled instruction using Microsoft Copilot. Both groups received didactic instruction, followed by illness script formulation tasks at three timepoints: pre-test (T1), immediate post-test (T2), and post-intervention with new cases (T3). Illness scripts were assessed using the validated Illness Script Assessment Tool (ISAT), which evaluates structure, content, richness, and maturity. Data was analyzed using cumulative link mixed-effects models.

Baseline illness script quality was comparable between groups. Chatbot-enabled instruction produced significantly greater improvements in script structure (OR 3.7), content (OR 3.3), and maturity (OR 2.9) at immediate post, with gains sustained at transfer testing for structure. Richness scores showed delayed improvement (OR 2.9). Composite ISAT scores confirmed durable improvements (OR 3.2 at T2; OR 2.7 at T3).

Chatbot-enabled instruction accelerated and enhanced illness script development compared with conventional methods, operationalizing deliberate practice principles and making non-analytical reasoning teachable. By integrating structured pedagogy with scalable AI feedback, this approach reduces educator burden while fostering diagnostic expertise. Future multi-site studies should examine long-term retention, clinical transfer, and AI feedback fidelity to ensure reliability and equity.