Exploring an LLM's Use in Supporting Journal Club Preparation and Discussion Among Residents

Journal clubs (JCs) play an important role in medical education by promoting critical appraisal and evidence-based practice. However, residents often face barriers to effective participation. Some of the issues that are commonly faced include limited time and difficulty understanding complex concepts and statistics.

This study was conducted during March-September 2024 and explored the use of a custom-trained Large Language Model (LLM) as a supportive tool for JC preparation and participation among postgraduate dental residents. Using a design-based research approach, researchers implemented the LLM integrated with relevant literature. Six JC sessions were conducted with sixteen residents across two subspecialties, accompanied by structured observations, feedback forms, and pre-/and post-focus groups with residents and faculty (n = 16).

Findings revealed that the LLM improved residents' comprehension of complex content, enhanced confidence, and increased engagement during discussions. Residents used the tool for summarizing articles, clarifying statistical methods, and generating discussion points. Fiftythree percent reported a positive experience of using the LLM for JC preparation, Forty-three percent were neutral, and only one response was negative. However, challenges included the need for precise prompt construction, occasional content inaccuracies, and limited depth in some specialized areas. Faculty observed enhanced participation but stressed the need for critical evaluation of LLM outputs. Both groups identified prompt-writing skills, critical thinking, and AI literacy as key competencies for effective LLM use.

LLMs can complement traditional teaching by supporting deeper engagement in JCs. As generative AI evolves, further research should examine its broader implications on learners' cognitive processes, epistemic trust, and educational equity.