利用专业放射科医生的专业知识来加强LLM对放射学报告的评估
Qingqing Zhu1, Xiuying Chen2, Qiao Jin1
1National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health, Bethesda, MD, USA.
ArXiv
|June 21, 2024
概括
本研究引入了一种使用大型语言模型 (LLM) 和放射科医生的专业知识来评估人工智能生成的放射学报告的新方法,提高了医疗人工智能评估的准确性和清晰度.
科学领域:
- 医学成像和人工智能 医学成像和人工智能
- 在医疗保健中的自然语言处理.
- 临床信息学 临床信息学
背景情况:
- 人工智能 (AI) 已经改善了放射学报告的生成,但自动评估方法是不够的.
- 像常规自然语言生成 (NLG) 和临床有效性 (CE) 等现有指标未能捕捉语义细微差别或在临床报告中优先考虑清晰度.
- 需要先进的评估指标,与专家放射科医生对人工智能生成的医疗报告的标准保持一致.
研究的目的:
- 开发和验证一种用于自动评估人工智能生成的放射学报告的新方法.
- 提高放射学自动化报告质量评估的准确性和临床相关性.
- 创建一个强大的系统,弥合AI能力和医疗报告中的专家人类判断之间的差距.
主要方法:
- 专业放射科医生专业知识与大型语言模型 (LLM) 的协同整合,特别是GPT-3.5和GPT-4.
- 应用语境指导学习 (ICIL) 和思维链 (CoT) 推理,使LLM评估与专家标准保持一致.
- 开发回归模型,以汇总句子级评估分数,用于全面报告评估.
主要成果:
- "详细的GPT-4 (5-shot) "模型的得分为0.48,比METEOR指标的得分高出0.19.
- "回归GPT-4"模型表现出与专家评估的优异对齐,超过现有指标0.35.
- 通过一项代策略,证实了评估解释的稳定性和有效性.
结论:
- 提出的基于LLM的方法,通过放射科医生输入和回归建模来增强,显著改善了人工智能生成的放射学报告的质量评估.
- 与传统指标相比,这种方法提供了更准确,更丰富的语义评估,解决了清晰度和临床背景的局限性.
- 计划发布的专家注释旨在为评估医疗报告中的AI建立一个新的基准,促进未来的进步.
相关概念视频
Improving Translational Accuracy
2.6K
2.6K
Radiological Investigation III: Pulmonary Angiogram and PET Scan
91
Radiological investigations are paramount in the diagnosis and management of various pulmonary diseases. Two essential investigations are the Pulmonary Angiogram and the Positron Emission Tomography (PET) Scan.
Pulmonary Angiogram
A Pulmonary Angiogram is an invasive procedure involving injecting a contrast medium through a catheter threaded into the pulmonary artery or the right side of the heart to visualize the pulmonary vasculature. Computed Tomography (CT) scans have mainly replaced this...
Pulmonary Angiogram
A Pulmonary Angiogram is an invasive procedure involving injecting a contrast medium through a catheter threaded into the pulmonary artery or the right side of the heart to visualize the pulmonary vasculature. Computed Tomography (CT) scans have mainly replaced this...
91
Radiological Investigation I: X-ray and CT
226
Radiological investigations, including X-rays and computed tomography (CT) scans, are critical for diagnosing and evaluating various medical conditions. These imaging techniques provide valuable insights into the body's internal structures, aiding in the detection of abnormalities, assessment of disease progression, and development of treatment strategies. This article delves into two primary radiological investigations, chest X-rays and CT scans, outlining their purpose, procedures, and...
226
Positron Emission Tomography
4.2K
Positron emission tomography (PET) is a medical imaging technique involving radiopharmaceuticals — substances that emit short-lived radiation. Although the first PET scanner was introduced in 1961, it took 15 more years before radiopharmaceuticals were combined with the technique and revolutionized its potential.
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body...
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body...
4.2K


