Related Experiment Video
Updated: Jun 4, 2025
Universal Screening for Prevention of Reading, Writing, and Math Disabilities in Spanish
Published on: July 18, 2020
Performance of ChatGPT in the Portuguese National Residency Access Examination
Gonçalo Ferraz-Costa1, Mafalda Griné2, Manuel Oliveira-Santos3
1Cardiology Department. Unidade Local de Saúde de Coimbra. Coimbra; Faculdade de Medicina. Universidade de Coimbra. Coimbra; Coimbra Institute for Clinical and Biomedical Research (iCBR). Coimbra. Portugal.
None:
ChatGPT, a language model developed by OpenAI, has been tested in several medical board examinations. This study aims to evaluate the performance of ChatGPT on the Portuguese National Residency Access Examination, a mandatory test for medical residency in Portugal. The study specifically compares the capabilities of ChatGPT versions 3.5 and 4o across five examination editions from 2019 to 2023. A total of 750 multiple-choice questions were submitted to both versions, and their answers were evaluated against the official responses. The findings revealed that ChatGPT 4o significantly outperformed ChatGPT 3.5, with a median examination score of 127 compared to 106 (p = 0.048). Notably, ChatGPT 4o achieved scores within the top 1% in two examination editions and exceeded the median performance of human candidates in all editions. Additionally, ChatGPT 4o's scores were high enough to qualify for any specialty. In conclusion, ChatGPT 4o can be a valuable tool for medical education and decision-making, but human oversight remains essential to ensure safe and accurate clinical practice.
