在耳鼻喉科的GPT-4o和o3-Mini的比较USMLE风格问题
Soumil Prasad1, Levi Travis, Mason Thornton
1University of Miami Miller School of Medicine, Miami, FL.
The Journal of craniofacial surgery
|September 8, 2025
概括
两个先进的AI语言模型,GPT-4o和o3-Mini,在耳鼻喉科USMLE样式问题上明显超过了医学学生,即使在先进的难度级别上也表现出高准确度.
科学领域:
- 医学教育 医学教育
- 人工智能在医学中的应用
- 耳鼻喉科 耳鼻喉科 耳鼻喉科
背景情况:
- 医学教育越来越多地利用人工智能工具进行学习.
- 对人类学习者进行AI性能评估对于评估教育实用性至关重要.
- 耳鼻喉科 (ENT) 培训需要掌握复杂,高风险的USMLE风格问题.
研究的目的:
- 将GPT-4o和o3-Mini的准确性与医学学生在耳鼻喉学专注的USMLE样式问题上的表现进行比较.
- 评估AI模型在不同问题难度级别的表现.
- 评估AI作为耳鼻喉科辅助学习工具的潜力.
主要方法:
- 从AMBOSS中提取了146个耳鼻喉学标记的USMLE第二步CK问题,按难度分层 (1-5).
- 通过API向GPT-4o和o3-Mini提出问题;将AI输出与历史的,身份不明的学生答案进行比较.
- 根据难度级别计算精度,并使用ANOVA和t测试进行统计分析 (α=0.05).
主要成果:
- 平均准确度:o3-Mini (93.35%) 和GPT-4o (90.45%),两者都明显优于学生 (55.44%).
- 人工智能模型在所有难度级别中都保持了≥86%的准确率,而学生的准确率从85.6% (一级) 降至26.7% (五级).
- o3-Mini在最困难的问题上 (第5级) 取得了100%的准确性.
结论:
- GPT-4o和o3-Mini在耳鼻喉科USMLE样式问题上表现优于平均医学学生.
- 人工智能模型保持高精度,即使在具有挑战性的内容上,也表明它们在医学培训中的价值.
- 这些发现支持将高级语言模型作为辅助学习资源纳入耳鼻喉学教育.
相关概念视频
01:13Multiple Comparison Tests
3.4K
Multiple comparison test, abbreviated as MCT, is a post hoc analysis generally performed after comparing multiple samples with one or more tests. An MCT will help identify a significantly different sample among multiple samples or a factor among multiple factors.
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
3.4K
01:26Sputum Studies I: Gram Stain, cytology, and Acid-fast smear and culture
1.3K
Sputum studies are a critical part of diagnosing and treating numerous respiratory conditions. These studies involve obtaining sputum samples for analysis to identify pathogenic organisms and assess the presence of abnormal cells indicative of malignant conditions. This lesson will delve into three fundamental sputum studies: Gram Stain, Cytology, and Acid-fast Smear and Culture.
Gram Stain
The Gram Stain is an integral part of sputum studies. It involves the staining of sputum, which permits...
Gram Stain
The Gram Stain is an integral part of sputum studies. It involves the staining of sputum, which permits...
1.3K
01:20Sputum Studies II: Culture and Sensitivity
2.0K
Description
Sputum culture and sensitivity is a medical procedure used to diagnose bacterial infections in the respiratory tract and select the most appropriate antibiotics for treatment. This process involves analyzing sputum samples of thick and opaque secretions produced in the lungs and airways. These samples are collected from patients and then sent to the laboratory for analysis.
The test can identify various pathogens responsible for respiratory infections, including Streptococcus,...
Sputum culture and sensitivity is a medical procedure used to diagnose bacterial infections in the respiratory tract and select the most appropriate antibiotics for treatment. This process involves analyzing sputum samples of thick and opaque secretions produced in the lungs and airways. These samples are collected from patients and then sent to the laboratory for analysis.
The test can identify various pathogens responsible for respiratory infections, including Streptococcus,...
2.0K
01:25Pulmonary Function Tests
1.2K
Pulmonary Function Tests (PFTs)
Pulmonary Function Tests are crucial diagnostic tools for assessing respiratory function, particularly in patients with chronic respiratory disorders. They comprehensively evaluate lung volumes, ventilatory function, breathing mechanics, diffusion, and gas exchange. These tests help diagnose pulmonary diseases and play a significant role in monitoring disease progression, evaluating disability, and assessing response to therapy.
PFTs involve using a spirometer, a...
Pulmonary Function Tests are crucial diagnostic tools for assessing respiratory function, particularly in patients with chronic respiratory disorders. They comprehensively evaluate lung volumes, ventilatory function, breathing mechanics, diffusion, and gas exchange. These tests help diagnose pulmonary diseases and play a significant role in monitoring disease progression, evaluating disability, and assessing response to therapy.
PFTs involve using a spirometer, a...
1.2K
01:30Serum Laboratory Studies, Stool Test, Breath Test
1.2K
Gastrointestinal (GI) diagnostic studies are pivotal in confirming, ruling out, diagnosing, or staging various diseases, including cancers. Following diagnosis, allocating time for discussions with the patient and providing informational resources is crucial. Diagnostic assessments of the GI tract often occur in outpatient settings like endoscopy suites or GI labs. Preparation for these tests may include dietary restrictions, fasting, liquid bowel preparations, laxatives, enemas, and the...
1.2K
01:24Automated Microbial Diagnostics
77
Automated diagnostic analyzers have transformed clinical microbiology by providing rapid and reliable methods for pathogen identification and antibiotic susceptibility testing. Among these systems, the Vitek 2 is widely used because it automates the traditionally labor-intensive processes of microbial identification (ID) and antibiotic susceptibility testing (AST), delivering standardized and timely results that are essential for effective patient care.Microbial Identification with ID CardsThe...
77