Designing Self-Directed Learning Content for Medical Students: An AI-Driven Evaluation of Learning Outcomes

The National Medical Commission of India requires self-directed learning (SDL) in the MBBS curriculum. SDL is a fundamental component of competency-based medical education (CBME). Although generative artificial intelligence (AI) technologies such as ChatGPT (OpenAI, San Francisco, CA) and Gemini (Google, Mountain View, CA) have expanded options for improving SDL, concerns about accuracy, academic integrity, and overreliance persist. This study assessed the quality, effectiveness, and learning outcomes of undergraduate medical students in relation to an organized AI-assisted SDL process.

Eighty-three second-year MBBS students from the Department of Microbiology at the Autonomous State Medical College in Kanpur Dehat participated in a single-center pre-post interventional trial using a within-subject crossover design. After structured instruction in prompt engineering, AI verification techniques, and a human-in-the-loop workflow, participants completed a standard SDL assignment, followed by an AI-assisted SDL assignment. A validated 10-domain rubric (maximum score: 20) covering accuracy, relevance, completeness, organization, clarity, clinical examples, visuals, critical thinking, appropriateness, and originality was used to evaluate assignment quality. Pre- and post-tests measured knowledge acquisition, and student opinions and task completion times were also recorded. Statistical analyses included Cohen's d effect sizes, paired t-tests, and the intraclass correlation coefficient (ICC) for inter-rater reliability.

In every rubric domain, AI-assisted SDL performed significantly better than traditional SDL (p < 0.001). Mean rubric scores improved by 29.1%, from 57.1% to 86.2% (Cohen's d = 2.8). Organization (+71.3%), completeness (+63.5%), and clarity (+62.5%) showed the largest gains. Knowledge scores increased from 5.2 to 8.8 out of 10 (p < 0.001; d = 2.5). Average task completion time decreased by 43%, from 44.5 to 25.4 minutes (p < 0.001; d = 1.5). Inter-rater reliability was excellent (ICC = 0.94). Most students reported feeling more confident (n = 65, 78%), synthesizing information more quickly (n = 76, 92%), and routinely using textbooks to verify AI-generated content (n = 71, 85%). The most commonly reported challenges were concerns about accuracy and timely formulation.

The systematic integration of generative AI into SDL significantly enhances MBBS students' learning efficiency, knowledge acquisition, and content quality. While highlighting the importance of maintaining human oversight to ensure accurate and responsible use of AI in medical education, our findings support integrating AI literacy and verification competencies into the medical curriculum.