一个临床上可访问的小型多模式放射学模型和评估指标,用于胸部X射线发现
Juan Manuel Zambrano Chaves1,2, Shih-Cheng Huang2, Yanbo Xu1
1Microsoft Research, Redmond, WA, USA.
Nature communications
|April 1, 2025
概括
开源的小型多式模式模型为放射学中的临床AI提供了具有成本效益的解决方案. LLaVA-Rad (7B) 在产生胸部X射线发现方面展示了最先进的性能,优于较大的模型.
科学领域:
- 人工智能的人工智能
- 医疗成像医学成像
- 放射学 放射学是一门学科.
背景情况:
- 生物医学中的大型基础模型面临临临床采用障碍,包括成本和评估可扩展性.
- 整合图像和文本的多式人工智能模型对于推进诊断能力至关重要.
研究的目的:
- 开发和评估一个开源的小型多式模式模型,用于从胸部X射线图像中生成自由文本的结果.
- 解决临床放射学AI中的性能差距,成本和可扩展性挑战.
主要方法:
- 一种以数据为中心的方法,使用697K精选的胸部X射线图像-文本对来训练一个域适应的编码器.
- 通过轻量级适配器将编码器与预先训练的语言模型集成在一起,以实现交叉模式对齐.
- 开发和验证CheXprompt,一种基于GPT-4的指标,用于评估与放射科医生评估的事实准确性.
主要成果:
- 专用多式模式模型LLaVA-Rad (7B) 在从胸部X射线中生成放射学报告方面取得了最先进的性能.
- 该模型在CheXprompt和其他事实性指标上表现明显优于更大的模型,包括GPT-4V和Med-PaLM M (84B).
- 该CheXprompt指标显示出强大的临床相关性,与放射科医生的评估保持一致.
结论:
- 开源的小型多式模式模型为放射学中的临床AI提供了一个可扩展,具有成本效益和保护隐私的替代方案.
- LLaVA-Rad代表了向可适应多式联络AI用于放射性诊断的重要一步,尽管需要进一步改进才能实现实时部署.
更多相关视频
10:26Author Spotlight: A 3D Digital Model for the Diagnosis and Treatment of Pulmonary Nodules
Published on: May 19, 2023
1.7K
11:31Using Micro-computed Tomography for the Assessment of Tumor Development and Follow-up of Response to Treatment in a Mouse Model of Lung Cancer
Published on: May 20, 2016
10.6K
相关概念视频
01:30Radiological Investigation I: X-ray and CT
185
Radiological investigations, including X-rays and computed tomography (CT) scans, are critical for diagnosing and evaluating various medical conditions. These imaging techniques provide valuable insights into the body's internal structures, aiding in the detection of abnormalities, assessment of disease progression, and development of treatment strategies. This article delves into two primary radiological investigations, chest X-rays and CT scans, outlining their purpose, procedures, and...
185
01:20Imaging Studies for Cardiovascular System III: X-Ray
116
The most common cardiovascular diagnostic test is an X-ray. It produces images of the heart, blood vessels, and adjacent structures.
Definition and Purpose
An X-ray, or radiograph, is a non-invasive method that uses ionizing radiation to take images of internal structures. It is mainly used in cardiac imaging to examine the heart, lungs, and major blood vessels, aiming to identify abnormalities in the heart's size, shape, and position, such as heart failure, congenital defects, and vascular...
Definition and Purpose
An X-ray, or radiograph, is a non-invasive method that uses ionizing radiation to take images of internal structures. It is mainly used in cardiac imaging to examine the heart, lungs, and major blood vessels, aiming to identify abnormalities in the heart's size, shape, and position, such as heart failure, congenital defects, and vascular...
116
01:24X-ray Imaging
5.2K
German physicist Wilhelm Röntgen (1845–1923) was experimenting with electrical current when he discovered that a mysterious and invisible "ray" would pass through his flesh but leave an outline of his bones on a screen coated with a metal compound. In 1895, Röntgen made the first durable record of the internal parts of a living human: an "X-ray" image (as it came to be called) of his wife’s hand. Scientists worldwide quickly began their own experiments with...
5.2K
01:26Flail Chest-II
148
Managing flail chest, a condition characterized by a segment of the chest wall moving independently from the rest of the thoracic cage, requires a comprehensive approach. It includes a thorough assessment of the patient's condition, a diagnostic evaluation to determine the extent of the injury, and the implementation of appropriate medical interventions tailored to the individual's needs.
Assessment:
1. Clinical Evaluation:
History:
Assessment:
1. Clinical Evaluation:
History:
148