Related Experiment Video
Updated: Jul 17, 2025

E-Patient Counseling Trial E-PACO: Computer Based Education versus Nurse Counseling for Patients to Prepare for Colonoscopy
Published on: August 1, 2019
Performance of ChatGPT in Israeli Hebrew OBGYN national residency examinations
Adiel Cohen1, Roie Alter2, Naama Lessans2
1Department of Obstetrics and Gynecology, Hadassah Medical Organization and Faculty of Medicine, Hebrew University of Jerusalem, Ein Kerem, P.O.B. 12000, 91120, Jerusalem, Israel. adielic@gmail.com.
Purpose:
Previous studies of ChatGPT performance in the field of medical examinations have reached contradictory results. Moreover, the performance of ChatGPT in other languages other than English is yet to be explored. We aim to study the performance of ChatGPT in Hebrew OBGYN-'Shlav-Alef' (Phase 1) examination.
Methods:
A performance study was conducted using a consecutive sample of text-based multiple choice questions, originated from authentic Hebrew OBGYN-'Shlav-Alef' examinations in 2021-2022. We constructed 150 multiple choice questions from consecutive text-based-only original questions. We compared the performance of ChatGPT performance to the real-life actual performance of OBGYN residents who completed the tests in 2021-2022. We also compared ChatGTP Hebrew performance vs. previously published English medical tests.
Results:
In 2021-2022, 27.8% of OBGYN residents failed the 'Shlav-Alef' examination and the mean score of the residents was 68.4. Overall, 150 authentic questions were evaluated (one examination). ChatGPT correctly answered 58 questions (38.7%) and reached a failed score. The performance of Hebrew ChatGPT was lower when compared to actual performance of residents: 38.7% vs. 68.4%, p < .001. In a comparison to ChatGPT performance in 9,091 English language questions in the field of medicine, the performance of Hebrew ChatGPT was lower (38.7% in Hebrew vs. 60.7% in English, p < .001).
Conclusions:
ChatGPT answered correctly on less than 40% of Hebrew OBGYN resident examination questions. Residents cannot rely on ChatGPT for the preparation of this examination. Efforts should be made to improve ChatGPT performance in other languages besides English.
More Related Videos
Related Concept Videos
Endoscopic Studies II: Thoracocentesis
Description
Excess pleural fluid or air may accumulate in some respiratory disorders in the thoracic cavity. To treat pleural effusion, a physician conducts thoracentesis by carefully piercing the chest wall and entering...
Assessment of the Gastrointestinal System II: Health Perception Pattern
Health Perception Patterns
Health perception patterns offer valuable insights into a patient's lifestyle habits and how they may impact their GI health. These patterns include:
Urologic Endoscopic Procedure: Cystoscopic Examination
Endoscopic Procedures V: ERCP
Patient...
Endoscopic Studies I: Bronchoscopy and Thoracoscopy
Bronchoscopy
Description
Bronchoscopy is a procedure that involves direct visualization of the larynx, trachea, and bronchi for diagnostic and therapeutic purposes. A flexible fiber optic or rigid bronchoscope is used to carry out the procedure. The fiber-optic bronchoscope is more frequently used due...
Endoscopic Procedures II: Colonoscopy

