ChatGPT-4 Performance on German Continuing Medical Education-Friend or Foe (Trick or Treat)? Protocol for a...

Area of Science:

• Medical Education
• Artificial Intelligence in Medicine
• Continuing Medical Education (CME)

Background:

• Artificial and assistive intelligence is expanding into applied medicine and continuing medical education (CME).
• CME is mandatory for physicians in Germany.
• This study investigates the use of AI, specifically large language models (LLMs) like ChatGPT-4, in CME.

Purpose of the Study:

• To determine if medical laypersons can pass CME tests with AI assistance.
• To explore AI applications in postgraduate medical education.
• To analyze the impact of LLMs on CME credit acquisition.

Main Methods:

• A randomized controlled trial was designed.
• Adult high school students attempted CME tests across six medical specialties.
• Three study arms involved varying levels of ChatGPT-4 usage.

Main Results:

• Data collection and student recruitment are ongoing as of October 2024.
• Preliminary findings are anticipated for publication in early 2025.
• The study is registered on the Open Science Framework (OSF).

Conclusions:

• AI, particularly LLMs like ChatGPT-4, may significantly influence medical laypersons' ability to pass CME tests.
• The findings suggest a need to re-evaluate the concept of continuous medical education.
• Potential implications for AI in medical training and assessment will be discussed.