ChatGPT-4 Performance on USMLE Step 1 Style Questions and Its Implications for Medical Education: A Comparative Study...

Area of Science:

• Medical Education
• Artificial Intelligence in Medicine

Background:

• The integration of artificial intelligence (AI) into medical education is a growing area of interest.
• Evaluating the capabilities of advanced AI models like ChatGPT-4 is crucial for understanding their potential applications.

Purpose of the Study:

• To assess the performance of OpenAI's ChatGPT-4 on United States Medical Licensing Exam (USMLE) Step 1 style questions.
• To explore the potential of AI as an interactive learning tool for preclinical medical students.

Main Methods:

• ChatGPT-4 was evaluated on 1300 USMLE Step 1 style questions covering various medical disciplines.
• Performance metrics were analyzed across different clinical domains.

Main Results:

• ChatGPT-4 achieved an accuracy rate of 86%, significantly surpassing the estimated 60% passing score.
• Performance was consistent across diverse medical topics, from biological processes to patient care ethics.
• The AI model demonstrated improvement over previous iterations.

Conclusions:

• ChatGPT-4 exhibits significant proficiency in answering USMLE Step 1 style questions.
• AI tools like ChatGPT-4 show promise as effective interactive learning aids for medical students.
• Further investigation into AI's role in preclinical medical education is warranted to enhance learning outcomes.