Related Experiment Video
Updated: Jun 9, 2025
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Performance of ChatGPT in medical licensing examinations in countries worldwide: A systematic review and
Mingxin Liu1, Tsuyoshi Okuhara2, Xinyi Chang3
1Department of Health Communication, Graduate School of Medicine, The University of Tokyo, Tokyo, Japan.
Introduction:
In November 2022, the online artificial intelligence (AI) chatbot ChatGPT was released to the public, and swiftly garnered global attention because of its ability to provide detailed answers to complex queries. In medical field, ChatGPT has shown great potential to be used in medical education and has excelled in many English-language medical licensing examinations. However, due to the variability of medical licensing examinations in different countries, and ChatGPT's particular proficiency in English, the previous literatures showed that ChatGPT is unable to pass medical licensing examinations from non-English-speaking countries or those not administered in English. To the best of our knowledge, this is the first study to review whether ChatGPT can demonstrate consistent accuracy across diverse medical licensing examinations and be used in medical education across countries.
Objective:
In this study protocol, we aimed to analyze and review the differences in performance of ChatGPT in medical exams in various language environments and countries, as well as its potential in medical education.
Methods And Analysis:
A systematic review and meta-analysis was conducted using PubMed, Web of Science, and Scopus to collect papers testing the performance of ChatGPT in medical licensing examinations. We imported all the collected literatures into Rayyan and screened the literatures based on the selection criteria and exclusion criteria. The risk of bias and quality of included studies was assessed by using Mixed Methods Appraisal Tool (MMAT). Data from included studies was extracted into an Excel spreadsheet. All of the above processes were completed by two reviewers independently. A third reviewer was consulted in cases of disagreement. Finally, we provided both quantitative and qualitative analysis of the findings from the included studies.
Trial Registration:
PROSPERO registration number: CRD42024506687.
Related Concept Videos
01:29Assessment of the Gastrointestinal System II: Health Perception Pattern
Health Perception Patterns
Health perception patterns offer valuable insights into a patient's lifestyle habits and how they may impact their GI health. These patterns include:
01:25Pulmonary Function Tests
Pulmonary Function Tests are crucial diagnostic tools for assessing respiratory function, particularly in patients with chronic respiratory disorders. They comprehensively evaluate lung volumes, ventilatory function, breathing mechanics, diffusion, and gas exchange. These tests help diagnose pulmonary diseases and play a significant role in monitoring disease progression, evaluating disability, and assessing response to therapy.
PFTs involve using a spirometer, a...
01:30Serum Laboratory Studies, Stool Test, Breath Test