对视觉语言模型 (GPT-4o和GPT-5) 的视觉解析和诊断推理的解:使用胸部成像测试案例进行分析
Dae Hee Han1, Eui Jin Hwang2, Soon Ho Yoon2,3
1Department of Radiology, Seoul St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Seoul, Korea.
视觉语言模型 (VLMs) 难以对胸部成像进行视觉分析,限制了诊断的准确性. 提供放射科医生生成的描述显著提高了VLM的性能,突显了人工智能辅助放射学中人类专业知识的需要.
科学领域:
- 人工智能在医学中的应用
- 医学成像分析 医学成像分析
- 放射学 放射学是一门学科.
背景情况:
- 视觉语言模型 (VLMs) 在分析放射图像和帮助诊断方面表现有前途.
- 目前的VLM表现不佳,阻碍了临床整合.
- 在视觉分析和诊断推理方面评估特定的VLM能力至关重要.
研究的目的:
- 评估视觉分析和诊断推理在基于 GPT 的 VLM 中对胸部成像诊断的独特贡献.
- 为了比较VLM性能与无放射科医生提供的图像描述.
主要方法:
- 来自韩国胸部成像协会问卷平台的128例胸部成像病例的回顾性分析.
- 两个VLM (GPT-4o,GPT-5) 处理了具有元数据和图像,或元数据和放射科医生生成的描述的病例.
- 评价前五个诊断准确度,图像总结质量和诊断与放射科医生表现的一致性.
主要成果:
- 在VLM中,描述与单独的图像 (15.9%-24.7%) 相比,VLM的前五名准确度更高 (40.1%-59.1%).
- 图像总结质量与两个模型的诊断准确性有显著的相关性.
- 使用图像描述 (78.8%-79.4%) 与直接图像输入 (31.6%-39.3%) 相比,使用专家放射科医生的诊断一致性要高得多.
结论:
- VLMs在视觉解释胸部成像结果方面表现出有限的能力.
- 当提供放射科医生生成的图像描述时,VLM诊断性能显著提高.
- 视觉图像解析,而不是诊断推理,是目前放射学VLM的主要限制.
更多相关视频
07:11Functional Magnetic Resonance Imaging (fMRI) of the Visual Cortex with Wide-View Retinotopic Stimulation
Published on: December 8, 2023
02:09Multi-modal Pulmonary Imaging: Using Complementary Information from CT and Hyperpolarized 129Xe MRI to Evaluate Lung Structure-Function
Published on: April 12, 2024
相关概念视频
Computed Tomography
The technique was invented in the 1970s and is based on the principle that as X-rays pass through the body, they are absorbed or reflected at different levels. In the technique, a patient lies on a motorized platform while a computerized axial tomography (CAT) scanner rotates...
Positron Emission Tomography
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body being...
Radiological Investigation II: MRI and Ventilation Perfusion Scan
Magnetic Resonance Imaging (MRI) and Ventilation Perfusion Scans are two radiological investigations that offer detailed diagnostic images of the body, particularly lung structures.
MRI
MRI uses magnetic fields and radiofrequency signals to distinguish between normal and abnormal tissues. This technology provides a more detailed diagnostic image than CT scans, enabling it to characterize pulmonary nodules, stage bronchogenic carcinoma, and evaluate inflammatory activity in...
Radiological Investigation III: Pulmonary Angiogram and PET Scan
Pulmonary Angiogram
A Pulmonary Angiogram is an invasive procedure involving injecting a contrast medium through a catheter threaded into the pulmonary artery or the right side of the heart to visualize the pulmonary vasculature. Computed Tomography (CT) scans have mainly replaced this...
Imaging Studies III: Computed Tomography
