聊天GPT在秘鲁国家执照医学考试上的表现:跨部门研究
Javier A Flores-Cohaila1,2, Abigaíl García-Vicente3,4, Sonia F Vizcarra-Jiménez4,5
1Academic Department, USAMEDIC, Lima, Peru.
聊天GPT (GPT-3.5和GPT-4) 在秘鲁国家医疗执照考试 (ENAM) 中表现出专家级别的表现,表现明显优于人类受试者. 快速工程提高了准确性,这表明在低收入国家提高医学教育的潜力.
科学领域:
- 人工智能在医学中的应用
- 医疗教育 技术 技术 医学教育
- 全球卫生平等全球卫生平等
背景情况:
- 在像USMLE这样的国际医疗执照考试中,ChatGPT表现出很高的表现.
- 对低收入国家的医疗执照考试中人工智能表现的研究有限.
- 秘鲁国家医疗执照考试的失败率很高,表明需要教育支持.
研究的目的:
- 评估GPT-3.5和GPT-4在秘鲁国家许可医疗检查 (ENAM) 的准确性.
- 识别导致ChatGPT提供不正确答案的因素.
- 探索用于提高ChatGPT性能的快速工程策略.
主要方法:
- 利用ENAM 2022数据集 (180个选择题) 来评估ChatGPT的准确性.
- 将ChatGPT的性能与1025名人类受试者的性能进行了比较.
- 分析了问题特征 (类型,难度,秘鲁特有的知识) 和影响错误答案的主题;采用多步提示来精确精确.
主要成果:
- GPT-4获得了86%的准确性,GPT-3.5获得了77%的准确性,明显超过了受试者平均水平的55%.
- 中等到高难度的问题与GPT-3.5 (OR 6.6) 和GPT-4 (OR 33.23) 的错误答案密切相关.
- 对最初不正确答案的提醒进行了改进,GPT-3.5的准确性从100%提高到29%不正确,GPT-4的准确性从100%提高到16%不正确.
结论:
- 聊天GPT (GPT-3.5和GPT-4) 在ENAM上展示了专家级的能力,超过了大多数人类测试者的表现.
- 问题难度是ChatGPT错误的一个关键因素,反映了人类的表现模式.
- 战略提示工程显著提高了ChatGPT的准确性,突出了它作为医学教育和考试准备的宝贵工具的潜力.
更多相关视频
05:04Author Spotlight: Evaluating Clinicians' Adoption of Ultrasound-Guided Vascular Cannulation Through Simulation Training
Published on: August 9, 2024
06:28E-Patient Counseling Trial E-PACO: Computer Based Education versus Nurse Counseling for Patients to Prepare for Colonoscopy
Published on: August 1, 2019
相关概念视频
Assessment of the Gastrointestinal System II: Health Perception Pattern
Health Perception Patterns
Health perception patterns offer valuable insights into a patient's lifestyle habits and how they may impact their GI health. These patterns include:
Pulmonary Function Tests
Pulmonary Function Tests are crucial diagnostic tools for assessing respiratory function, particularly in patients with chronic respiratory disorders. They comprehensively evaluate lung volumes, ventilatory function, breathing mechanics, diffusion, and gas exchange. These tests help diagnose pulmonary diseases and play a significant role in monitoring disease progression, evaluating disability, and assessing response to therapy.
PFTs involve using a spirometer, a...
Pre-Procedural Guidelines for Assessing Blood Pressure
Radiological Investigation III: Pulmonary Angiogram and PET Scan
Pulmonary Angiogram
A Pulmonary Angiogram is an invasive procedure involving injecting a contrast medium through a catheter threaded into the pulmonary artery or the right side of the heart to visualize the pulmonary vasculature. Computed Tomography (CT) scans have mainly replaced this...
Serum Laboratory Studies, Stool Test, Breath Test
