评估ChatGPT在放射学子专业的表现:对板式检查准确性和可变性的元分析
Dan Nguyen1, Grace Hyun J Kim2, Arash Bedayat3
1University of Massachusetts Chan Medical School, Worcester, MA, United States of America.
Clinical imaging
|June 25, 2025
概括
大型语言模型显示放射学委员会问题准确率为58.83%,GPT-4的表现优于GPT-3.5.5. 性能因子专业和输入类型而异,表明需要优化.
科学领域:
- 人工智能在医学中的应用
- 医学成像和诊断 医学成像和诊断
背景情况:
- 像ChatGPT这样的大型语言模型 (LLM) 在医疗保健中越来越多地用于信息合成和临床决策支持.
- 之前的研究评估了LLM在一般医学考试中的表现,但放射学特定数据,特别是关于快速策略和输入方式的数据仍然有限.
研究的目的:
- 这一元分析评估了ChatGPT在放射学委员会样式问题上的表现.
- 该研究评估了各种放射学子专业的准确性,提示工程技术,GPT模型版本和输入方式.
主要方法:
- 在PubMed和SCOPUS的系统文献搜索中,发现了163篇文章,其中16篇符合纳入标准.
- 提取的数据包括子专业,准确性,问题数量,GPT模型,输入模式和提示策略.
- 统计分析涉及两个比例的z测试,二项式通用线性模型 (GLM) 和具有随机效应的元回归.
主要成果:
- 在7024个问题中,整体准确率为58.83%,根据子专业有显著的变化 (例如,急诊放射学73.00%与肌肉骨放射学49.24%).
- GPT-4和GPT-4o显著超过了GPT-3.5的表现 (p <.001).
- 文本输入 (67.10%) 的准确性高于视觉输入 (46.52%,p < .001),基本提示提高了性能 (p < .01). 随着时间的推移,人们注意到性能下降 (p < .001).
结论:
- 聊天GPT在回答放射学板式问题方面显示出潜力,但表现不一致.
- 局限性包括视觉推理的挑战和快速工程和研究方法的变化.
- 需要有针对性的优化,以提高LLM在放射学中的实用性.
更多相关视频
相关概念视频
01:13Radiological Investigation III: Pulmonary Angiogram and PET Scan
174
Radiological investigations are paramount in the diagnosis and management of various pulmonary diseases. Two essential investigations are the Pulmonary Angiogram and the Positron Emission Tomography (PET) Scan.
Pulmonary Angiogram
A Pulmonary Angiogram is an invasive procedure involving injecting a contrast medium through a catheter threaded into the pulmonary artery or the right side of the heart to visualize the pulmonary vasculature. Computed Tomography (CT) scans have mainly replaced this...
Pulmonary Angiogram
A Pulmonary Angiogram is an invasive procedure involving injecting a contrast medium through a catheter threaded into the pulmonary artery or the right side of the heart to visualize the pulmonary vasculature. Computed Tomography (CT) scans have mainly replaced this...
174
01:30Radiological Investigation II: MRI and Ventilation Perfusion Scan
234
Description
Magnetic Resonance Imaging (MRI) and Ventilation Perfusion Scans are two radiological investigations that offer detailed diagnostic images of the body, particularly lung structures.
MRI
MRI uses magnetic fields and radiofrequency signals to distinguish between normal and abnormal tissues. This technology provides a more detailed diagnostic image than CT scans, enabling it to characterize pulmonary nodules, stage bronchogenic carcinoma, and evaluate inflammatory activity in...
Magnetic Resonance Imaging (MRI) and Ventilation Perfusion Scans are two radiological investigations that offer detailed diagnostic images of the body, particularly lung structures.
MRI
MRI uses magnetic fields and radiofrequency signals to distinguish between normal and abnormal tissues. This technology provides a more detailed diagnostic image than CT scans, enabling it to characterize pulmonary nodules, stage bronchogenic carcinoma, and evaluate inflammatory activity in...
234
01:30Radiological Investigation I: X-ray and CT
431
Radiological investigations, including X-rays and computed tomography (CT) scans, are critical for diagnosing and evaluating various medical conditions. These imaging techniques provide valuable insights into the body's internal structures, aiding in the detection of abnormalities, assessment of disease progression, and development of treatment strategies. This article delves into two primary radiological investigations, chest X-rays and CT scans, outlining their purpose, procedures, and...
431
01:27Imaging Studies III: Computed Tomography
56
DefinitionComputed Tomography (CT) of the genitourinary (GU) tract is a non-invasive imaging modality that utilizes X-rays and computer processing to generate detailed cross-sectional images of the urinary system, encompassing the kidneys, ureters, bladder, and adjacent structures such as the adrenal glands.PurposeCT scans of the GU tract serve several diagnostic and therapeutic purposes, including:Diagnosis of Urinary Tract Diseases: Detects kidney stones, tumors, cysts, and congenital...
56