进入大型多式联络模型时代:对ChatGPT-4V (视觉) 解释放射图像的能力进行试点研究
Lingxuan Zhu1,2, Weiming Mou3, Yancheng Lai1
1Department of Oncology, Zhujiang Hospital, Southern Medical University, Guangzhou.
International journal of surgery (London, England)
|March 18, 2024
概括
像ChatGPT-4V这样的大型多式模式模型在解释放射图像和帮助手术规划方面表现有前途. 这种人工智能工具可以诊断疾病,并提供患者病史和成像数据时建议治疗计划.
科学领域:
- 人工智能在医学中的应用
- 医学成像分析 医学成像分析
- 大型多式联运模型 (LMM)
背景情况:
- 聊天GPT-4V的"聊天用图像"功能的出现标志着大型多式模式模型 (LMM) 的新时代.
- 放射学解读对于手术规划和术后护理至关重要,但LMM在这个领域的能力需要彻底评估.
- LMMs有可能彻底改变手术团队使用放射数据的潜力是显著的.
研究的目的:
- 为了全面评估ChatGPT-4V在解释放射图像方面的性能.
- 评估ChatGPT-4V在基于放射数据的基础上制定临床治疗计划的能力.
- 确定患者病史对ChatGPT-4V.诊断准确性的影响.
主要方法:
- 仅使用图像的87个USMLE样式问题评估诊断能力.
- 评估了使用五分利克尔特尺度的治疗计划配方.
- 测试了具有或没有详细病史的诊断性能.
- 从胸部X射线8分析了100张胸部X射线,以检测异常.
- 检查了ChatGPT-4V在复杂病例上的表现,来自放射学杂志的"诊断请"部分.
主要成果:
- 聊天GPT-4V在USMLE样式的历史问题上实现了77.01%的诊断准确率.
- 治疗计划的平均得分为3.97,表明表现良好.
- 在没有病史的情况下,诊断准确度显著下降至19.54% (P<0.0001).
- 在检测胸部放射学异常时,获得了0.768的AUC,但在没有病史的情况下,具体诊断是有限的.
- 复杂病例的诊断与专家的答案一致.
结论:
- 聊天GPT-4V有效地整合了患者病史和放射性图像,用于诊断和治疗计划.
- 该模型显示了在放射学和外科护理中临床应用的巨大潜力.
- 患者病史对于医学成像解释中LMMs的准确诊断性能至关重要.
相关概念视频
01:13Radiological Investigation III: Pulmonary Angiogram and PET Scan
92
Radiological investigations are paramount in the diagnosis and management of various pulmonary diseases. Two essential investigations are the Pulmonary Angiogram and the Positron Emission Tomography (PET) Scan.
Pulmonary Angiogram
A Pulmonary Angiogram is an invasive procedure involving injecting a contrast medium through a catheter threaded into the pulmonary artery or the right side of the heart to visualize the pulmonary vasculature. Computed Tomography (CT) scans have mainly replaced this...
Pulmonary Angiogram
A Pulmonary Angiogram is an invasive procedure involving injecting a contrast medium through a catheter threaded into the pulmonary artery or the right side of the heart to visualize the pulmonary vasculature. Computed Tomography (CT) scans have mainly replaced this...
92
01:29Positron Emission Tomography
4.2K
Positron emission tomography (PET) is a medical imaging technique involving radiopharmaceuticals — substances that emit short-lived radiation. Although the first PET scanner was introduced in 1961, it took 15 more years before radiopharmaceuticals were combined with the technique and revolutionized its potential.
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body...
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body...
4.2K
01:25Imaging Studies II: Positron Emission Tomography and Scintigraphy
113
Positron Emission Tomography (PET) is a medical imaging technique that provides crucial insights into the body's physiological functions at a molecular level. It is an indispensable resource for diagnosing, staging, and monitoring various illnesses, notably cancer, neurological disorders, and cardiovascular conditions.
Fundamental Principles of PET
Fundamental Principles of PET
113
01:30Radiological Investigation II: MRI and Ventilation Perfusion Scan
119
Description
Magnetic Resonance Imaging (MRI) and Ventilation Perfusion Scans are two radiological investigations that offer detailed diagnostic images of the body, particularly lung structures.
MRI
MRI uses magnetic fields and radiofrequency signals to distinguish between normal and abnormal tissues. This technology provides a more detailed diagnostic image than CT scans, enabling it to characterize pulmonary nodules, stage bronchogenic carcinoma, and evaluate inflammatory activity in...
Magnetic Resonance Imaging (MRI) and Ventilation Perfusion Scans are two radiological investigations that offer detailed diagnostic images of the body, particularly lung structures.
MRI
MRI uses magnetic fields and radiofrequency signals to distinguish between normal and abnormal tissues. This technology provides a more detailed diagnostic image than CT scans, enabling it to characterize pulmonary nodules, stage bronchogenic carcinoma, and evaluate inflammatory activity in...
119