Related Experiment Video
Updated: Aug 6, 2026
06:23
The 4 Mountains Test: A Short Test of Spatial Memory with High Sensitivity for the Diagnosis of Pre-dementia Alzheimer's Disease
Published on: October 13, 2016
The temporal changes in GPT-4 performance on UKMLA practice questions: educational and clinical implications
Ravanth Baskaran1,2, Sai Sirikonda3, Aditya Singh2
1University Hospital Southampton NHS Trust, Southampton, United Kingdom.
Frontiers in Medicine
|August 5, 2026
Summary
Generative Artificial Intelligence (GAI) model GPT-4 showed improved performance on the United Kingdom Medical Licensing Assessment (UKMLA) over 18 months. This suggests vendor updates enhance clinical reasoning, positioning GPT-4 as a potential medical education tool.
Area of Science:
- Artificial Intelligence in Medicine
- Medical Education Technology
- Large Language Models
Background:
- Generative Artificial Intelligence (GAI) models like GPT-4 are increasingly studied for medical applications.
- GPT-4 demonstrates high performance on medical licensing exams, including the UKMLA.
- Longitudinal data on GPT-4's performance stability and evolution over time is limited.
Purpose of the Study:
- To analyze the time-spaced performance of GPT-4 on the UKMLA.
- To characterize how GPT-4's performance changes over time due to vendor updates.
- To assess the impact of iterative model improvements on clinical reasoning.
Main Methods:
- Two publicly available UKMLA exams from June 2023 and December 2024 were used.
- GPT-4 answered 191 questions, both with and without multiple-choice options.
- McNemar's test was applied to evaluate performance changes over time.
Main Results:
- GPT-4 accuracy improved significantly between test periods (MCQ: 88.0% to 93.7%; non-MCQ: 68.1% to 81.7%).
- Both single-step and multi-step question accuracy increased, with MCQ formats consistently performing better.
- Performance gains were observed across all question categories, notably in management questions without MCQ options.
Conclusions:
- GPT-4's performance on the UKMLA significantly improved over 18 months, indicating enhanced clinical reasoning from vendor updates.
- GPT-4 shows potential as a supplementary educational tool for medical students and clinicians.
- The study highlights the need for structured oversight when deploying AI tools in medicine due to opaque performance drivers.
