Related Experiment Video
Updated: Jul 2, 2025
E-Patient Counseling Trial E-PACO: Computer Based Education versus Nurse Counseling for Patients to Prepare for Colonoscopy
Published on: August 1, 2019
Performance of ChatGPT in Israeli Hebrew Internal Medicine National Residency Exam
David J Ozeri1, Adiel Cohen2, Noa Bacharach3
1Division of Rheumatology, Sheba Medical Center, Tel Hashomer, Israel.
Background:
Completing internal medicine specialty training in Israel involves passing the Israel National Internal Medicine Exam (Shlav Aleph), a challenging multiple-choice test. multiple-choice test. Chat generative pre-trained transformer (ChatGPT) 3.5, a language model, is increasingly used for exam preparation.
Objectives:
To assess the ability of ChatGPT 3.5 to pass the Israel National Internal Medicine Exam in Hebrew.
Methods:
Using the 2023 Shlav Aleph exam questions, ChatGPT received prompts in Hebrew. Textual questions were analyzed after the appeal, comparing its answers to the official key.
Results:
ChatGPT 3.5 correctly answered 36.6% of the 133 analyzed questions, with consistent performance across topics, except for challenges in nephrology and biostatistics.
Conclusions:
While ChatGPT 3.5 has excelled in English medical exams, its performance in the Hebrew Shlav Aleph was suboptimal. Factors include limited training data in Hebrew, translation complexities, and unique language structures. Further investigation is essential for its effective adaptation to Hebrew medical exam preparation.
Related Concept Videos
01:29Assessment of the Gastrointestinal System II: Health Perception Pattern
Health Perception Patterns
Health perception patterns offer valuable insights into a patient's lifestyle habits and how they may impact their GI health. These patterns include:
01:26Endoscopic Studies II: Thoracocentesis
Description
Excess pleural fluid or air may accumulate in some respiratory disorders in the thoracic cavity. To treat pleural effusion, a physician conducts thoracentesis by carefully piercing the chest wall and entering...
01:30Serum Laboratory Studies, Stool Test, Breath Test