胸部放射病例的大型语言模型和一般放射科医生的诊断性能:一项比较研究
Yasin Celal Gunes1, Turay Cesur2
1Department of Radiology, Kirikkale Yuksek Ihtisas Hospital, Kirikkale.
Journal of thoracic imaging
|September 13, 2024
概括
与其他大型语言模型 (LLM) 和人类放射科医生相比,Claude 3 Opus在胸部放射病例中表现出卓越的诊断准确性. 这项研究强调了LLM在临床决策支持系统中的潜力,并提供了适当的医疗监督.
科学领域:
- 医疗成像中的人工智能
- 胸部放射学 诊断 诊断 胸部放射学
- 在医疗保健中的自然语言处理.
背景情况:
- 大型语言模型 (LLM) 正在越来越多地探索其在医学诊断中的潜力.
- 胸部放射学提出了复杂的诊断挑战,人工智能工具可以提供支持.
- 对LLM与专家放射科医生的诊断性能进行评估对于临床整合至关重要.
研究的目的:
- 为了比较十种不同的大型语言模型 (LLM) 与两位经过董事会认证的一般放射科医生的诊断性能.
- 评估诊断胸部放射病例的表现,来源于胸部放射学学会.
- 分析各种LLM和人类专家的诊断准确度和差异诊断得分 (DDxScore).
主要方法:
- 利用了来自胸部放射学学会 (2012-2023) 的124份"本月的案例"出版物.
- 士获得了病史和成像检查结果,用于诊断和差异诊断.
- 放射科医生提供独立的视觉评估;诊断准确性和DDxScore被统计分析.
主要成果:
- 克劳德3Opus实现了最高的诊断准确度 (70.29%),超过了其他LLMs和放射科医生.
- 与大多数LLM和放射科医生相比,Claude 3 Opus也表现出明显更好的差异诊断得分 (DDxScore).
- 在特定的放射性诊断中,LLM和放射科医生都表现更好,在大多数解剖学子组中没有显著差异.
结论:
- 与其他LLM和放射科医生相比,Claude 3 Opus在基于文本的胸部放射病例诊断中表现优异.
- 作为放射学临床决策支持工具,LLM显著有前途,但需要适当的医疗监督.
- 对医学诊断中LLM能力的进一步研究是有必要的.
更多相关视频
相关概念视频
01:30Radiological Investigation I: X-ray and CT
218
Radiological investigations, including X-rays and computed tomography (CT) scans, are critical for diagnosing and evaluating various medical conditions. These imaging techniques provide valuable insights into the body's internal structures, aiding in the detection of abnormalities, assessment of disease progression, and development of treatment strategies. This article delves into two primary radiological investigations, chest X-rays and CT scans, outlining their purpose, procedures, and...
218
01:15Receiver Operating Characteristic Plot
101
A ROC (Receiver Operating Characteristic) plot is a graphical tool used to assess the performance of a binary classification model by illustrating the trade-off between sensitivity (true positive rate) and specificity (false positive rate). By plotting sensitivity against 1 - specificity across various threshold settings, the ROC curve shows how well the model distinguishes between classes, with a curve closer to the top-left corner indicating a more accurate model. The area under the ROC curve...
101
01:13Radiological Investigation III: Pulmonary Angiogram and PET Scan
88
Radiological investigations are paramount in the diagnosis and management of various pulmonary diseases. Two essential investigations are the Pulmonary Angiogram and the Positron Emission Tomography (PET) Scan.
Pulmonary Angiogram
A Pulmonary Angiogram is an invasive procedure involving injecting a contrast medium through a catheter threaded into the pulmonary artery or the right side of the heart to visualize the pulmonary vasculature. Computed Tomography (CT) scans have mainly replaced this...
Pulmonary Angiogram
A Pulmonary Angiogram is an invasive procedure involving injecting a contrast medium through a catheter threaded into the pulmonary artery or the right side of the heart to visualize the pulmonary vasculature. Computed Tomography (CT) scans have mainly replaced this...
88
01:30Radiological Investigation II: MRI and Ventilation Perfusion Scan
105
Description
Magnetic Resonance Imaging (MRI) and Ventilation Perfusion Scans are two radiological investigations that offer detailed diagnostic images of the body, particularly lung structures.
MRI
MRI uses magnetic fields and radiofrequency signals to distinguish between normal and abnormal tissues. This technology provides a more detailed diagnostic image than CT scans, enabling it to characterize pulmonary nodules, stage bronchogenic carcinoma, and evaluate inflammatory activity in...
Magnetic Resonance Imaging (MRI) and Ventilation Perfusion Scans are two radiological investigations that offer detailed diagnostic images of the body, particularly lung structures.
MRI
MRI uses magnetic fields and radiofrequency signals to distinguish between normal and abnormal tissues. This technology provides a more detailed diagnostic image than CT scans, enabling it to characterize pulmonary nodules, stage bronchogenic carcinoma, and evaluate inflammatory activity in...
105