评估GPT-V4 (具有视觉的GPT-4) 关于检测胸部X射线图上的放射性发现
Yiliang Zhou1, Hanley Ong1, Patrick Kennedy1
1From the Departments of Population Health Sciences (Y.Z., Y.P.) and Radiology (H.O., P.K., J.K., K.H., G.S.), Weill Cornell Medicine, 425 E 61st St, Ste 301, New York, NY 10065; Department of Thoracic Imaging, University of Texas MD Anderson Cancer Center, Houston, Tex (C.C.W.); and Department of Radiology, Thomas Jefferson University Hospital, Philadelphia, Pa (A.F.).
Radiology
|May 7, 2024
概括
预先训练的生成变压器,视觉GPT-4 (GPT-4V),在解释现实世界胸部X射线图以产生放射性发现方面表现出有限的有效性. 虽然几次射击学习提高了性能,但GPT-4V提高了性能.
科学领域:
- 医疗成像中的人工智能
- 用于放射学的自然语言处理.
- 计算机视觉在医疗保健中的应用
背景情况:
- 从胸部X射线图中自动生成放射性发现对于医学图像分析至关重要.
- 开放AI的视觉GPT-4 (GPT-4V) 具有自动图像-文本对生成的潜力.
- 在现实世界胸部放射学中GPT-4V的有效性需要彻底调查.
研究的目的:
- 评估GPT-4V在从现实世界胸部X射线图中产生放射性发现的能力.
- 将GPT-4V的性能与放射科医生注释的参考标准进行比较.
- 在零射击和少数射击学习环境中评估绩效.
主要方法:
- 来自NIH和MIDRC数据集的100张胸部X射线成像的回顾性研究.
- 放射科医生注释为临床条件和具有横向性的ICD-10代码建立了一个参考标准.
- 在零射击和少数射击设置中评估的GPT-4V性能,与参考标准相比.
主要成果:
- 在零射击设置中,GPT-4V在ICD-10代码检测方面实现了较低的性能指标 (例如,NIH的F1得分为7.3%,MIDRC的18.2%).
- 包括横向性进一步降低了性能 (例如,NIH的F1得分为4.5%,MIDRC的6.4%).
- 短暂的学习表明,真正阳性率和F1分数有所改善,但积极的预测值并没有大幅增加.
结论:
- 尽管在自然图像理解方面有希望,但GPT-4V在解释现实世界胸部X射线图方面表现出有限的有效性.
- 目前的性能表明,GPT-4V还不适合从胸部X射线中自动生成放射性发现.
- 需要进一步的研究和模型开发,以增强医疗图像分析中的AI能力.
更多相关视频
相关概念视频
01:13Radiological Investigation III: Pulmonary Angiogram and PET Scan
91
Radiological investigations are paramount in the diagnosis and management of various pulmonary diseases. Two essential investigations are the Pulmonary Angiogram and the Positron Emission Tomography (PET) Scan.
Pulmonary Angiogram
A Pulmonary Angiogram is an invasive procedure involving injecting a contrast medium through a catheter threaded into the pulmonary artery or the right side of the heart to visualize the pulmonary vasculature. Computed Tomography (CT) scans have mainly replaced this...
Pulmonary Angiogram
A Pulmonary Angiogram is an invasive procedure involving injecting a contrast medium through a catheter threaded into the pulmonary artery or the right side of the heart to visualize the pulmonary vasculature. Computed Tomography (CT) scans have mainly replaced this...
91
01:30Radiological Investigation I: X-ray and CT
232
Radiological investigations, including X-rays and computed tomography (CT) scans, are critical for diagnosing and evaluating various medical conditions. These imaging techniques provide valuable insights into the body's internal structures, aiding in the detection of abnormalities, assessment of disease progression, and development of treatment strategies. This article delves into two primary radiological investigations, chest X-rays and CT scans, outlining their purpose, procedures, and...
232
01:30Radiological Investigation II: MRI and Ventilation Perfusion Scan
114
Description
Magnetic Resonance Imaging (MRI) and Ventilation Perfusion Scans are two radiological investigations that offer detailed diagnostic images of the body, particularly lung structures.
MRI
MRI uses magnetic fields and radiofrequency signals to distinguish between normal and abnormal tissues. This technology provides a more detailed diagnostic image than CT scans, enabling it to characterize pulmonary nodules, stage bronchogenic carcinoma, and evaluate inflammatory activity in...
Magnetic Resonance Imaging (MRI) and Ventilation Perfusion Scans are two radiological investigations that offer detailed diagnostic images of the body, particularly lung structures.
MRI
MRI uses magnetic fields and radiofrequency signals to distinguish between normal and abnormal tissues. This technology provides a more detailed diagnostic image than CT scans, enabling it to characterize pulmonary nodules, stage bronchogenic carcinoma, and evaluate inflammatory activity in...
114
01:29Positron Emission Tomography
4.2K
Positron emission tomography (PET) is a medical imaging technique involving radiopharmaceuticals — substances that emit short-lived radiation. Although the first PET scanner was introduced in 1961, it took 15 more years before radiopharmaceuticals were combined with the technique and revolutionized its potential.
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body...
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body...
4.2K
01:26Pulmonary Tuberculosis IV
139
Tuberculosis, more commonly referred to as TB, is an infectious disease stemming from Mycobacterium tuberculosis. While it primarily impacts the lungs, TB can also affect other body areas. Given its severity and global impact, timely and accurate diagnosis is crucial for controlling its spread and improving patient outcomes.
Several diagnostic approaches are used to detect TB. The conventional method is the Tuberculin Skin Test (TST), also known as the Mantoux test. However, this method has...
Several diagnostic approaches are used to detect TB. The conventional method is the Tuberculin Skin Test (TST), also known as the Mantoux test. However, this method has...
139
01:25Imaging Studies II: Positron Emission Tomography and Scintigraphy
111
Positron Emission Tomography (PET) is a medical imaging technique that provides crucial insights into the body's physiological functions at a molecular level. It is an indispensable resource for diagnosing, staging, and monitoring various illnesses, notably cancer, neurological disorders, and cardiovascular conditions.
Fundamental Principles of PET
Fundamental Principles of PET
111