Related Experiment Video
Updated: Jun 12, 2025
05:04
Author Spotlight: Evaluating Clinicians' Adoption of Ultrasound-Guided Vascular Cannulation Through Simulation Training
Published on: August 9, 2024
872
ChatGPT's Performance on Iran's Medical Licensing Exams
Alireza Keshtkar1, Ali-Asghar Hayat2, Farnaz Atighi3
1Research Center of Noncommunicable Diseases, Jahrom University of Medical Sciences, Jahrom, Iran.
Medical Journal of the Islamic Republic of Iran
|June 9, 2025
Summary
ChatGPT demonstrates potential in medical education, scoring above passing thresholds on Iranian medical license exams in basic sciences and pre-internship levels. The AI model also provided reasoning, highlighting its utility for medical training and assessment.
Area of Science:
- Artificial Intelligence in Medicine
- Natural Language Processing Applications
- Medical Education Technology
Background:
- OpenAI's ChatGPT, a 175 billion parameter transformer model, is utilized for natural language processing tasks.
- This study evaluates ChatGPT's knowledge and interpretive capabilities on Iranian medical licensing examinations.
Purpose of the Study:
- To assess the performance of ChatGPT (specifically ChatGPT 3.5) on Iranian medical licensing exams.
- To compare ChatGPT's performance across different exam levels (basic sciences, pre-internship, pre-residency) and languages (Persian and English).
Main Methods:
- Three levels of Iranian medical license exams were administered to ChatGPT 3.5.
- Exams were presented in both Persian and English, with versions reflecting ChatGPT's internet access.
- Accuracy and concordance of responses were evaluated by two blinded adjudicators.
Main Results:
- ChatGPT answered 48.5% of 2210 total questions accurately, with 91% overall concordance.
- English questions yielded higher accuracy (61.4%) and concordance (94.5%) than Persian questions (35.7% accuracy, 88.7% concordance).
- The model exceeded passing scores for basic sciences and pre-internship exams and met the minimum for residency application.
Conclusions:
- ChatGPT demonstrates proficiency exceeding passing standards in certain Iranian medical licensing exams.
- The AI's ability to provide reasoning suggests significant potential for integration into medical education.
- Performance variations between English and Persian indicate language as a critical factor in AI assessment accuracy.
Related Concept Videos
01:25Pulmonary Function Tests
311
Pulmonary Function Tests (PFTs)
Pulmonary Function Tests are crucial diagnostic tools for assessing respiratory function, particularly in patients with chronic respiratory disorders. They comprehensively evaluate lung volumes, ventilatory function, breathing mechanics, diffusion, and gas exchange. These tests help diagnose pulmonary diseases and play a significant role in monitoring disease progression, evaluating disability, and assessing response to therapy.
PFTs involve using a spirometer, a...
Pulmonary Function Tests are crucial diagnostic tools for assessing respiratory function, particularly in patients with chronic respiratory disorders. They comprehensively evaluate lung volumes, ventilatory function, breathing mechanics, diffusion, and gas exchange. These tests help diagnose pulmonary diseases and play a significant role in monitoring disease progression, evaluating disability, and assessing response to therapy.
PFTs involve using a spirometer, a...
311
01:29Assessment of the Gastrointestinal System II: Health Perception Pattern
81
Assessing the gastrointestinal (GI) system is a complex process that begins with collecting subjective data. This data, collected through patient interviews, provides crucial insights into the patient's health history, perception patterns, and lifestyle habits, all contributing significantly to GI health.
Health Perception Patterns
Health perception patterns offer valuable insights into a patient's lifestyle habits and how they may impact their GI health. These patterns include:
Health Perception Patterns
Health perception patterns offer valuable insights into a patient's lifestyle habits and how they may impact their GI health. These patterns include:
81
01:18Assessment of the Abdomen II: Percussion
212
Percussion is a fundamental technique used to assess the liver, spleen, and abdominal organs by tapping the abdomen and interpreting the resulting sounds. This method helps identify fluid, distention, and masses through variations in sound, such as the high-pitched tympany of air-filled areas and the dullness of solid masses. Understanding how to percuss these organs provides valuable information for healthcare professionals in diagnosing conditions early.
Percussion
Percussion is an essential...
Percussion
Percussion is an essential...
212
01:25Imaging Studies II: Positron Emission Tomography and Scintigraphy
91
Positron Emission Tomography (PET) is a medical imaging technique that provides crucial insights into the body's physiological functions at a molecular level. It is an indispensable resource for diagnosing, staging, and monitoring various illnesses, notably cancer, neurological disorders, and cardiovascular conditions.
Fundamental Principles of PET
Fundamental Principles of PET
91
01:13Radiological Investigation III: Pulmonary Angiogram and PET Scan
86
Radiological investigations are paramount in the diagnosis and management of various pulmonary diseases. Two essential investigations are the Pulmonary Angiogram and the Positron Emission Tomography (PET) Scan.
Pulmonary Angiogram
A Pulmonary Angiogram is an invasive procedure involving injecting a contrast medium through a catheter threaded into the pulmonary artery or the right side of the heart to visualize the pulmonary vasculature. Computed Tomography (CT) scans have mainly replaced this...
Pulmonary Angiogram
A Pulmonary Angiogram is an invasive procedure involving injecting a contrast medium through a catheter threaded into the pulmonary artery or the right side of the heart to visualize the pulmonary vasculature. Computed Tomography (CT) scans have mainly replaced this...
86