Transforming medical education: leveraging large language models to enhance PBL-a proof-of-concept study

The alignment of learning materials with learning objectives (LOs) is critical for successfully implementing the problem-based learning (PBL) curriculum. This study investigated the capabilities of Gemini Advanced, a large language model (LLM), in creating clinical vignettes that align with LOs and comprehensive tutor guides. This study used a faculty-written clinical vignette about diabetes mellitus for third-year medical students. We submitted the LOs and the associated clinical vignette and tutor guide to the LLM to evaluate their alignment and generate new versions. Four faculty members compared both versions, using a structured questionnaire. The mean evaluation scores for original and LLM-generated versions are reported. The LLM identified new triggers for the clinical vignette to align it better with the LOs. Moreover, it restructured the tutor guide for better organization and flow and included thought-provoking questions. The medical information provided by the LLM was scientifically appropriate and accurate. The LLM-generated clinical vignette scored higher (3.0 vs. 1.25) for alignment with the LOs. However, the original version scored better for being educational level-appropriate (2.25 vs. 1.25) and adhering to PBL design (2.50 vs. 1.25). The LLM-generated tutor guide scored higher for better flow (3.0 vs. 1.25), comprehensive and relevant content (2.75 vs. 1.50), and thought-provoking questions (2.25 vs. 1.75). However, LLM-generated learning material lacked visual elements. In conclusion, this study demonstrated that Gemini could align and improve PBL learning materials. By leveraging the potential of LLMs while acknowledging their limitations, medical educators can create innovative and effective learning experiences for future physicians.NEW & NOTEWORTHY This study evaluated a large language model (LLM) (Gemini Advanced) for creating aligned problem-based learning (PBL) materials. The LLM improved the alignment of the clinical vignette with learning goals. The LLM also restructured the tutor guide and added thought-provoking questions. The LLM guide was well organized and informative, but the original vignette was considered more educational level-appropriate. Although the LLM could not generate visuals, AI can improve PBL materials, especially when combined with human expertise.