聊天GPT在医学专业考试中的表现:一项观察性研究
Ayse Dilara Oztermeli1, Ahmet Oztermeli2
1Emergency Medicine Department, Gebze Fatih State Hospital, Kocaeli, Turkey.
Medicine
|August 11, 2023
概括
在医学专业考试中,ChatGPT表现出适度的成功,达到高达70.9%的准确性. 虽然它还没有达到专家级别,但它的表现表明了医疗保健应用的潜力.
科学领域:
- 人工智能在医学中的应用
- 医疗教育 技术 技术 医学教育
- 自然语言处理自然语言处理.
背景情况:
- 人工智能 (AI) 在医学教育和实践中的整合正在迅速发展.
- 像ChatGPT这样的大型语言模型在知识评估和医疗保健服务中提供了潜在的应用.
- 在高风险的医学检查中评估AI性能对于了解其当前的能力和局限性至关重要.
研究的目的:
- 评估ChatGPT (GPT-3.5) 在最近的医学专业考试 (MSE) 上的表现.
- 为了确定ChatGPT在这些考试中在人类候选人中潜在的排名.
- 根据其考试成绩,探索ChatGPT在医疗保健服务中的实用性.
主要方法:
- 利用了最近5个中小企业公开提供的问题和答案密钥,共计1177个问题.
- 将这些问题管理到ChatGPT GPT-3.5版本 (2023年3月23日).
- 根据平均分数,排名和问题类型 (短/长,单选/多选) 分析表现.
主要成果:
- 聊天GPT取得的成功率从54.3%到70.9%的正确答案.
- 它的排名各不相同,最好的表现是在22214名候选人中排名第1787位.
- 与他们的同行相比,在短问题 (P=.03) 和单选MCQ (P<.001) 中观察到的准确性明显更高.
结论:
- 在具有挑战性的医学专业考试中,ChatGPT表现出了显著的成功,这表明其潜在的实用性.
- 目前的表现表明,它落后于人类专家,但对未来的发展充满希望.
- 人工智能在医疗评估和服务中的作用需要继续进行研究和开发.
相关概念视频
01:29Assessment of the Gastrointestinal System II: Health Perception Pattern
141
Assessing the gastrointestinal (GI) system is a complex process that begins with collecting subjective data. This data, collected through patient interviews, provides crucial insights into the patient's health history, perception patterns, and lifestyle habits, all contributing significantly to GI health.
Health Perception Patterns
Health perception patterns offer valuable insights into a patient's lifestyle habits and how they may impact their GI health. These patterns include:
Health Perception Patterns
Health perception patterns offer valuable insights into a patient's lifestyle habits and how they may impact their GI health. These patterns include:
141
01:26Imaging Studies III: Gastrointestinal Motility Studies and Virtual Colonoscopy
109
This lesson explores three gastrointestinal imaging techniques: radionuclide testing, colonic transit studies, and virtual colonoscopy.
Radionuclide Testing
Radionuclide testing is a sophisticated medical technique for assessing gastrointestinal motility. It focuses on gastric emptying and colonic transit time. Radioactive markers track the movement of food through the digestive system, providing insights into gastrointestinal disorders.
In gastric emptying studies, a meal's liquid and...
Radionuclide Testing
Radionuclide testing is a sophisticated medical technique for assessing gastrointestinal motility. It focuses on gastric emptying and colonic transit time. Radioactive markers track the movement of food through the digestive system, providing insights into gastrointestinal disorders.
In gastric emptying studies, a meal's liquid and...
109
01:05Data Collection III
2.8K
The physical assessment examines the patient for objective data that defines the patient's condition, and aids in formulating the nursing care plan. The purpose of physical assessment is a health status appraisal, which includes identifying health problems, and establishing a database for nursing intervention.
The principles to begin the physical assessment include conducting a comprehensive or problem-related history in a quiet, well-lit room, emphasizing privacy and comfort for the...
The principles to begin the physical assessment include conducting a comprehensive or problem-related history in a quiet, well-lit room, emphasizing privacy and comfort for the...
2.8K
01:13Radiological Investigation III: Pulmonary Angiogram and PET Scan
115
Radiological investigations are paramount in the diagnosis and management of various pulmonary diseases. Two essential investigations are the Pulmonary Angiogram and the Positron Emission Tomography (PET) Scan.
Pulmonary Angiogram
A Pulmonary Angiogram is an invasive procedure involving injecting a contrast medium through a catheter threaded into the pulmonary artery or the right side of the heart to visualize the pulmonary vasculature. Computed Tomography (CT) scans have mainly replaced this...
Pulmonary Angiogram
A Pulmonary Angiogram is an invasive procedure involving injecting a contrast medium through a catheter threaded into the pulmonary artery or the right side of the heart to visualize the pulmonary vasculature. Computed Tomography (CT) scans have mainly replaced this...
115