Related Experiment Video
Updated: May 21, 2025
Translational Brain Mapping at the University of Rochester Medical Center: Preserving the Mind Through Personalized Brain Mapping
Published on: August 12, 2019
Is artificial intelligence successful in the Turkish neurology board exam?
Ayse Betul Acar1, Ece Yanik2, Emine Altin3
1Departments of Neurology and Algology, Etlik City Hospital, Ankara, Turkiye.
Objectives:
OpenAI declared that GPT-4 performed better in academic and certain specialty areas. Medical licensing exams assess the clinical competence of doctors. We aimed to investigate for the first time howChatGPT will perform in the Turkish Neurology Proficiency Exam.
Methods:
TheGPT-4 version of ChatGPT was used in the study due to the presence ofimage-based questions. The multiple-choice sections of the TurkishNeurology Proficiency Exams conducted by the Turkish NeurologyAssociation (TND) in 2021, 2022 and 2023 were applied to ChatGPT-4.Questions and multiple-choice answers were used in their originalTurkish forms in the official national examination standards.
Results:
The success rate in all three exams ranges from 79% to 82%. There were common and different mistakes in the two trials. When the incorrect answers were re-evaluated, the correct answers were obtained.
Conclusions:
This is the first study to investigate the performance of ChatGPT on the real Neurology Proficiency Examination. The success rate was shown to be above GPT-3.5. Furthermore, this study showed that translating questions from the original language into English did not affect the performance of GPT-4 in medical licensing exams, unlike GPT-3.5. It is therefore very important that the information obtained is accurate and verifiable. ChatGPT-4's ability to find the correct answer with feedback on questions that it initially answered incorrectly may be due to the model's ability to generate flexible and adaptive answers.These models should be used carefully and consciously, knowing that they will not always give the correct answer.
