医学学生的放射学考试中的大型语言模型 (LLMs):性能和后果
Jennifer Gotta1, Quang Anh Le Hong1, Vitali Koch1
1Department of Diagnostic and Interventional Radiology, Goethe University Frankfurt, Frankfurt am Main, Germany.
概括
像ChatGPT这样的大型语言模型 (LLM) 在医学教育中表现有前途,GPT-4在医学学生的放射学考试中获得了88.1%的准确性. 虽然对审查有用,但用户必须意识到潜在的不准确性.
科学领域:
- 医疗教育 技术 技术 医学教育
- 放射学中的人工智能
- 自然语言处理应用程序
背景情况:
- 医疗教育正随着技术整合而迅速发展.
- 大型语言模型 (LLM) 提供了简化复杂医疗概念和增强交互式学习的潜力.
- 法学士已经在专业考试中表现出熟练,这表明对医学等专业领域的相关性.
研究的目的:
- 评估LLMs在专为医学学生设计的放射学考试中的表现.
- 评估LLM在特定医疗领域的现有能力和影响.
- 在放射学问题上比较不同LLM版本 (GPT-3.5,GPT-4) 和平台 (Perplexity AI) 的准确性.
主要方法:
- 使用了一套来自医学学生放射学考试的151个多选题.
- 问题按类型和主题分类.
- 通过API使用OpenAI的GPT-3.5和GPT-4,以及手动使用PerplexityAI (GPT-3.5和Bing) 来评估LLM的性能.
主要成果:
- GPT-4实现了88.1%的整体准确性,显著超过了GPT-3.5 (67.6%) (p<0.001).
- 与GPT-3.5,困惑AI和医学学生相比,GPT-4在低级和高级问题上表现出卓越的表现.
- 所有测试的GPT模型都将通过大学医学学生的放射学考试.
结论:
- 士学位,特别是GPT-4,显示出作为医学学生审查放射学考试内容的可访问知识资源的潜力.
- 在低级和高级问题上,GPT-4的强表现表明它作为学习工具的实用性.
- 放射科医生和学生应该意识到LLM的局限性,例如倾向于提供自信但不正确的答案.
相关概念视频
Positron Emission Tomography
4.0K
Positron emission tomography (PET) is a medical imaging technique involving radiopharmaceuticals — substances that emit short-lived radiation. Although the first PET scanner was introduced in 1961, it took 15 more years before radiopharmaceuticals were combined with the technique and revolutionized its potential.
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body...
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body...
4.0K
Radiological Investigation I: X-ray and CT
214
Radiological investigations, including X-rays and computed tomography (CT) scans, are critical for diagnosing and evaluating various medical conditions. These imaging techniques provide valuable insights into the body's internal structures, aiding in the detection of abnormalities, assessment of disease progression, and development of treatment strategies. This article delves into two primary radiological investigations, chest X-rays and CT scans, outlining their purpose, procedures, and...
214
Imaging Studies II: Positron Emission Tomography and Scintigraphy
88
Positron Emission Tomography (PET) is a medical imaging technique that provides crucial insights into the body's physiological functions at a molecular level. It is an indispensable resource for diagnosing, staging, and monitoring various illnesses, notably cancer, neurological disorders, and cardiovascular conditions.
Fundamental Principles of PET
Fundamental Principles of PET
88
Radiological Investigation III: Pulmonary Angiogram and PET Scan
86
Radiological investigations are paramount in the diagnosis and management of various pulmonary diseases. Two essential investigations are the Pulmonary Angiogram and the Positron Emission Tomography (PET) Scan.
Pulmonary Angiogram
A Pulmonary Angiogram is an invasive procedure involving injecting a contrast medium through a catheter threaded into the pulmonary artery or the right side of the heart to visualize the pulmonary vasculature. Computed Tomography (CT) scans have mainly replaced this...
Pulmonary Angiogram
A Pulmonary Angiogram is an invasive procedure involving injecting a contrast medium through a catheter threaded into the pulmonary artery or the right side of the heart to visualize the pulmonary vasculature. Computed Tomography (CT) scans have mainly replaced this...
86
Radiological Investigation II: MRI and Ventilation Perfusion Scan
100
Description
Magnetic Resonance Imaging (MRI) and Ventilation Perfusion Scans are two radiological investigations that offer detailed diagnostic images of the body, particularly lung structures.
MRI
MRI uses magnetic fields and radiofrequency signals to distinguish between normal and abnormal tissues. This technology provides a more detailed diagnostic image than CT scans, enabling it to characterize pulmonary nodules, stage bronchogenic carcinoma, and evaluate inflammatory activity in...
Magnetic Resonance Imaging (MRI) and Ventilation Perfusion Scans are two radiological investigations that offer detailed diagnostic images of the body, particularly lung structures.
MRI
MRI uses magnetic fields and radiofrequency signals to distinguish between normal and abnormal tissues. This technology provides a more detailed diagnostic image than CT scans, enabling it to characterize pulmonary nodules, stage bronchogenic carcinoma, and evaluate inflammatory activity in...
100
Imaging Studies for Cardiovascular System III: X-Ray
147
The most common cardiovascular diagnostic test is an X-ray. It produces images of the heart, blood vessels, and adjacent structures.
Definition and Purpose
An X-ray, or radiograph, is a non-invasive method that uses ionizing radiation to take images of internal structures. It is mainly used in cardiac imaging to examine the heart, lungs, and major blood vessels, aiming to identify abnormalities in the heart's size, shape, and position, such as heart failure, congenital defects, and vascular...
Definition and Purpose
An X-ray, or radiograph, is a non-invasive method that uses ionizing radiation to take images of internal structures. It is mainly used in cardiac imaging to examine the heart, lungs, and major blood vessels, aiming to identify abnormalities in the heart's size, shape, and position, such as heart failure, congenital defects, and vascular...
147


