Exploring an LLM's Use in Supporting Journal Club Preparation and Discussion Among Residents.

Area of Science:

• Medical Education Technology
• Artificial Intelligence in Healthcare
• Dental Postgraduate Training

Background:

• Journal clubs (JCs) are vital for medical education, fostering critical appraisal and evidence-based practice.
• Residents face barriers like time constraints and complex material comprehension in JCs.
• Effective participation in JCs is crucial for professional development.

Purpose of the Study:

• To explore the utility of a custom-trained Large Language Model (LLM) as a supportive tool for postgraduate dental residents' journal club (JC) preparation and participation.
• To assess the impact of LLM integration on resident engagement, comprehension, and confidence during JCs.
• To identify challenges and necessary competencies for effective LLM use in medical education.

Main Methods:

• A design-based research approach was used from March-September 2024.
• A custom-trained LLM was integrated with literature for six JC sessions involving sixteen postgraduate dental residents.
• Data collection included observations, feedback forms, and pre-/post-focus groups with residents and faculty.

Main Results:

• LLM use improved residents' understanding of complex topics, boosted confidence, and increased discussion engagement.
• Residents utilized the LLM for article summarization, statistical method clarification, and generating discussion points.
• While 53% reported positive experiences, challenges included prompt specificity, occasional inaccuracies, and the need for critical evaluation of AI outputs.

Conclusions:

• Large Language Models can effectively supplement traditional teaching methods to enhance engagement in journal clubs.
• Further research is needed on the broader implications of generative AI on learners' cognition, trust, and educational equity.
• Developing AI literacy and critical evaluation skills is essential for maximizing the benefits of LLMs in medical education.