雷克萨明-全球:揭示放射学报告生成指标不一致性的框架
Oishi Banerjee1, Agustina Saenz2, Kay Wu3
1Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA, oishi_banerjee@g.harvard.edu.
Pacific Symposium on Biocomputing. Pacific Symposium on Biocomputing
|December 13, 2024
概括
对于放射学报告,需要新的AI评估指标. ReXamine-Global是一个大型语言模型 (LLM) 支持的框架,测试了医院内的现有指标,揭示了显著的概括性差距.
科学领域:
- 医疗成像中的人工智能
- 放射学报告的生成和评估
- 在医疗保健中的自然语言处理.
背景情况:
- 放射学生成性AI模型正在迅速发展,需要强大的方法来评估人工智能生成的报告质量.
- 评估人工智能放射学报告的现有指标可能缺乏跨不同临床环境和报告风格的概括性.
- 一个标准化的多站点框架对于验证AI报告评估指标至关重要.
研究的目的:
- 开发和应用一个多站点框架,ReXamine-Global,以测试已建立的AI放射学报告评估指标的通用性.
- 评估对报告风格变化的度量敏感性以及不同医院站点对专家评估的同意.
- 确定当前指标的局限性,并指导开发更强大的AI报告评估工具.
主要方法:
- 开发了ReXamine-Global,这是一个大型语言模型 (LLM) 支持的框架,用于人工智能报告评估指标的多站点测试.
- 评估了对人工智能生成和实地真相报告之间的风格差异的度量敏感性.
- 用240份报告对6家国际医院的AI生成报告质量进行专家评估.
主要成果:
- 将ReXamine-Global应用于七个已建立的指标显示,在各种医院环境中,它们的概括性存在重大差距.
- 某些指标对报告风格表现出不良的敏感性,导致不一致的质量评分.
- 在特定地点观察到指标得分和专家意见之间的差异,突出显示了可靠性问题.
结论:
- 人工智能生成的放射学报告的现有指标在不同机构的概括性方面存在严重的限制.
- ReXamine-Global为开发人员设计和验证强大的AI评估指标提供了一个至关重要的工具.
- 这些发现指导用户选择适合其特定医院环境的适当指标和评估程序.
相关概念视频
Imaging Studies for Cardiovascular System III: X-Ray
143
The most common cardiovascular diagnostic test is an X-ray. It produces images of the heart, blood vessels, and adjacent structures.
Definition and Purpose
An X-ray, or radiograph, is a non-invasive method that uses ionizing radiation to take images of internal structures. It is mainly used in cardiac imaging to examine the heart, lungs, and major blood vessels, aiming to identify abnormalities in the heart's size, shape, and position, such as heart failure, congenital defects, and vascular...
Definition and Purpose
An X-ray, or radiograph, is a non-invasive method that uses ionizing radiation to take images of internal structures. It is mainly used in cardiac imaging to examine the heart, lungs, and major blood vessels, aiming to identify abnormalities in the heart's size, shape, and position, such as heart failure, congenital defects, and vascular...
143
X-ray Imaging
5.4K
German physicist Wilhelm Röntgen (1845–1923) was experimenting with electrical current when he discovered that a mysterious and invisible "ray" would pass through his flesh but leave an outline of his bones on a screen coated with a metal compound. In 1895, Röntgen made the first durable record of the internal parts of a living human: an "X-ray" image (as it came to be called) of his wife’s hand. Scientists worldwide quickly began their own experiments with...
5.4K
Imaging Studies for Cardiovascular System IV: CMRI
5
Cardiovascular magnetic resonance imaging, or CMRI, is a non-invasive diagnostic test that employs a magnetic field and radiofrequency waves to create precise images of the heart and arteries. It provides comprehensive information about cardiac anatomy, function, perfusion, and tissue characterization without ionizing radiation.IndicationsCMRI diagnoses various heart conditions, including tissue damage from heart attacks, ischemic heart disease, myocarditis, aortic issues (tears, aneurysms,...
5
Radiological Investigation I: X-ray and CT
209
Radiological investigations, including X-rays and computed tomography (CT) scans, are critical for diagnosing and evaluating various medical conditions. These imaging techniques provide valuable insights into the body's internal structures, aiding in the detection of abnormalities, assessment of disease progression, and development of treatment strategies. This article delves into two primary radiological investigations, chest X-rays and CT scans, outlining their purpose, procedures, and...
209
Positron Emission Tomography
4.0K
Positron emission tomography (PET) is a medical imaging technique involving radiopharmaceuticals — substances that emit short-lived radiation. Although the first PET scanner was introduced in 1961, it took 15 more years before radiopharmaceuticals were combined with the technique and revolutionized its potential.
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body...
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body...
4.0K
Imaging Studies for Cardiovascular System VI: Calcium -Scoring CT
3
Calcium-Scoring CT ScanA calcium-scoring CT scan, also known as coronary artery calcium (CAC) scan, detects calcium deposits in the coronary arteries. This test assesses the risk of coronary artery disease (CAD), which can lead to cardiovascular events such as angina, heart failure, and sudden cardiac arrest.A calcium-scoring CT scan is generally recommended for individuals at intermediate risk of CAD without symptoms. It includes:Men aged 40-75 and women aged 50-75: Especially those with a...
3


