大型语言模型与视觉上的诊断放射学委员会考试风格问题
Shawn H Sun1, Kasha Chen1, Samuel Anavim1
1University of California Irvine, Radiology Department, UCI Medical Center, Orange, California, USA.
Academic radiology
|December 4, 2024
概括
有视觉能力的大型语言模型在使用图像时没有提高放射学考试的表现. 克劳德3.5索内特在仅文本输入方面表现出色,在这个特定的放射学AI应用中表现优于GPT-4V和Gemini 1.5 Pro.
科学领域:
- 放射学中的人工智能
- 医学成像分析 医学成像分析
- 大型语言模型
背景情况:
- 大型语言模型 (LLM) 越来越能够处理视觉数据,在放射学等医疗领域提出潜在的应用.
- 评估这些多式联络LLM在专业领域的有效性对于理解它们的实际实用性至关重要.
研究的目的:
- 评估当代多式联运大型语言模型在放射学委员会式考试问题上的表现.
- 为了比较仅文本,仅图像和多式联络提示对这些AI模型在放射学环境中的有效性.
主要方法:
- 从AuntMinnie公共测试银行回顾选择了280个放射学板式问题.
- 将问题转换为纯文本,纯图像和多式联机提示格式,用于三个LLM:GPT-4V,Gemini 1.5 Pro和Claude 3.5 Sonnet.
- 使用科克兰Q测试和双对麦克纳马测试进行统计分析,以比较模型和提示格式的性能.
主要成果:
- GPT-4V和Gemini 1.5 Pro在文本,图像或多模式提示符中没有显著的性能差异.
- 克劳德3.5 索内特的性能显著下降,与图像输入相比,只有文本和多式输入.
- 克劳德3.5索内特在纯文本和多式联机格式中显著超过了GPT-4V和Gemini 1.5 Pro.
结论:
- 目前可视化的大型语言模型并不能通过图像利用来提高放射学板式问题的性能.
- 克劳德3.5 索内特在纯文本输入方面表现出卓越的性能,这表明LLM在放射学任务方面的能力有了显著的进步.
相关概念视频
01:30Radiological Investigation II: MRI and Ventilation Perfusion Scan
99
Description
Magnetic Resonance Imaging (MRI) and Ventilation Perfusion Scans are two radiological investigations that offer detailed diagnostic images of the body, particularly lung structures.
MRI
MRI uses magnetic fields and radiofrequency signals to distinguish between normal and abnormal tissues. This technology provides a more detailed diagnostic image than CT scans, enabling it to characterize pulmonary nodules, stage bronchogenic carcinoma, and evaluate inflammatory activity in...
Magnetic Resonance Imaging (MRI) and Ventilation Perfusion Scans are two radiological investigations that offer detailed diagnostic images of the body, particularly lung structures.
MRI
MRI uses magnetic fields and radiofrequency signals to distinguish between normal and abnormal tissues. This technology provides a more detailed diagnostic image than CT scans, enabling it to characterize pulmonary nodules, stage bronchogenic carcinoma, and evaluate inflammatory activity in...
99
01:30Radiological Investigation I: X-ray and CT
209
Radiological investigations, including X-rays and computed tomography (CT) scans, are critical for diagnosing and evaluating various medical conditions. These imaging techniques provide valuable insights into the body's internal structures, aiding in the detection of abnormalities, assessment of disease progression, and development of treatment strategies. This article delves into two primary radiological investigations, chest X-rays and CT scans, outlining their purpose, procedures, and...
209
01:13Radiological Investigation III: Pulmonary Angiogram and PET Scan
86
Radiological investigations are paramount in the diagnosis and management of various pulmonary diseases. Two essential investigations are the Pulmonary Angiogram and the Positron Emission Tomography (PET) Scan.
Pulmonary Angiogram
A Pulmonary Angiogram is an invasive procedure involving injecting a contrast medium through a catheter threaded into the pulmonary artery or the right side of the heart to visualize the pulmonary vasculature. Computed Tomography (CT) scans have mainly replaced this...
Pulmonary Angiogram
A Pulmonary Angiogram is an invasive procedure involving injecting a contrast medium through a catheter threaded into the pulmonary artery or the right side of the heart to visualize the pulmonary vasculature. Computed Tomography (CT) scans have mainly replaced this...
86
01:29Positron Emission Tomography
4.0K
Positron emission tomography (PET) is a medical imaging technique involving radiopharmaceuticals — substances that emit short-lived radiation. Although the first PET scanner was introduced in 1961, it took 15 more years before radiopharmaceuticals were combined with the technique and revolutionized its potential.
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body...
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body...
4.0K
01:20Imaging Studies for Cardiovascular System III: X-Ray
143
The most common cardiovascular diagnostic test is an X-ray. It produces images of the heart, blood vessels, and adjacent structures.
Definition and Purpose
An X-ray, or radiograph, is a non-invasive method that uses ionizing radiation to take images of internal structures. It is mainly used in cardiac imaging to examine the heart, lungs, and major blood vessels, aiming to identify abnormalities in the heart's size, shape, and position, such as heart failure, congenital defects, and vascular...
Definition and Purpose
An X-ray, or radiograph, is a non-invasive method that uses ionizing radiation to take images of internal structures. It is mainly used in cardiac imaging to examine the heart, lungs, and major blood vessels, aiming to identify abnormalities in the heart's size, shape, and position, such as heart failure, congenital defects, and vascular...
143
01:28Imaging Studies for Cardiovascular System V: CT
3
Cardiac computed tomography (CT) scanning is an advanced cardiac imaging technique that utilizes CT technology, with or without intravenous (IV) contrast, to produce accurate cross-sectional virtual slices of specific areas of the heart, coronary circulation, and major blood vessels such as the aorta, pulmonary veins, and arteries. The computer processes these slices to generate three-dimensional images. Multidetector CT (MDCT) is a rapid form of CT scanning that captures multiple slices...
3