在放射学中评估视觉语言模型的临床信息框架报告生成:错误分类学和风险意识度量表
Hao Guan1,2, Peter C Hou1,2, Pengyu Hong3
1Brigham and Women's Hospital, Boston, MA.
medRxiv : the preprint server for health sciences
|August 12, 2025
概括
这项研究引入了一个新的框架来评估人工智能生成的放射学报告,重点关注临床风险和安全. 它识别了视觉语言模型 (VLMs) 中常见的错误,以提高诊断准确性.
科学领域:
- 人工智能的人工智能
- 医疗成像医学成像
- 自然语言处理自然语言处理.
背景情况:
- 视觉语言模型 (VLMs) 显示了自动放射学报告生成的前景.
- 这些报告的现有评估指标不足,缺乏临床特异性.
- 需要强有力的评估方法,考虑患者安全和临床相关性.
研究的目的:
- 为VLM生成的放射学报告开发一个临床信息的评估框架.
- 引入一种新的风险意识指标来评估人工智能生成的报告的安全影响.
- 分析目前放射学领域领先的VLM的性能,并确定其漏洞.
主要方法:
- 定义了12种放射学特异性错误类型的分类,由医生标注临床风险水平 (低,中,高).
- 在685个专家注释的MIMIC-CXR案例中对三个VLM (DeepSeek VL2,CXR-LLaVA,CheXagent) 进行了全面的错误分析.
- 引入了文本生成临床风险加权错误评分 (CREST) 度量来量化安全影响.
主要成果:
- 在评估的VLM中确定了关键模型漏洞和常见错误模式.
- 揭示了与不同类型的错误相关的特定条件的风险概况.
- 证明了传统NLP指标在捕捉临床上显著的不准确性方面的局限性.
结论:
- 拟议的框架为评估医疗报告生成模型提供了一个以安全为中心的基础.
- 研究结果为开发和部署更可靠的放射学AI工具提供了可操作的见解.
- 在临床应用中,CREST的度量和错误分类学可以指导未来的VLM性能改进.
更多相关视频
03:14Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
681
07:08Using Optical Coherence Tomography and Optokinetic Response As Structural and Functional Visual System Readouts in Mice and Rats
Published on: January 10, 2019
10.1K
相关概念视频
01:30Radiological Investigation I: X-ray and CT
414
Radiological investigations, including X-rays and computed tomography (CT) scans, are critical for diagnosing and evaluating various medical conditions. These imaging techniques provide valuable insights into the body's internal structures, aiding in the detection of abnormalities, assessment of disease progression, and development of treatment strategies. This article delves into two primary radiological investigations, chest X-rays and CT scans, outlining their purpose, procedures, and...
414
01:30Radiological Investigation II: MRI and Ventilation Perfusion Scan
222
Description
Magnetic Resonance Imaging (MRI) and Ventilation Perfusion Scans are two radiological investigations that offer detailed diagnostic images of the body, particularly lung structures.
MRI
MRI uses magnetic fields and radiofrequency signals to distinguish between normal and abnormal tissues. This technology provides a more detailed diagnostic image than CT scans, enabling it to characterize pulmonary nodules, stage bronchogenic carcinoma, and evaluate inflammatory activity in...
Magnetic Resonance Imaging (MRI) and Ventilation Perfusion Scans are two radiological investigations that offer detailed diagnostic images of the body, particularly lung structures.
MRI
MRI uses magnetic fields and radiofrequency signals to distinguish between normal and abnormal tissues. This technology provides a more detailed diagnostic image than CT scans, enabling it to characterize pulmonary nodules, stage bronchogenic carcinoma, and evaluate inflammatory activity in...
222