放射学中的视觉大语言模型:对诊断准确性和幻觉的系统多模型评估
Marc Sebastian von der Stück1, Roman Vuskov1, Simon Westfechtel1
1Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, 52074 Aachen, Germany.
Life (Basel, Switzerland)
|January 28, 2026
概括
视觉大语言模型 (VLLMs) 在放射学中显示出有限的诊断可靠性,精度和频繁的幻觉存在显著差异. 临床背景可以提高性能,但不能克服目前可靠的医学图像解释的局限性.
科学领域:
- 人工智能在医学中的应用
- 医学成像分析 医学成像分析
- 放射学 放射学是一门学科.
背景情况:
- 视觉大语言模型 (VLLMs) 正在成为帮助放射科医生解释医疗图像的潜在工具.
- 然而,它们在放射学中的实际临床实用性和诊断性能在很大程度上尚未得到证实,需要进行彻底的调查.
研究的目的:
- 系统地比较放射学中通用和生物医学VLLM的诊断性能.
- 评估临床背景和成像方式对VLLM准确性和幻觉率的影响.
主要方法:
- 在180张临床图像 (60张X射线图,60张CT图,60张MRI图) 上对7个VLLM (ChatGPT-4o,Gemini 2.0,Claude Sonnet 3.7,Perplexity AI,Google Vision AI,LLaVA-1.6,LLaVA-Med-v1.5) 的评估,并进行了验证诊断.
- 模型解释图像,并没有提供临床背景.
- 混合效应逻辑回归分析了跨模型,模式和背景的诊断准确性和幻觉频率.
主要成果:
- 在VLLM中观察到诊断准确度的显著变化 (8.1%-29.2%),Gemini 2.0表现最好,LLaVA表现最差.
- 计算机断层扫描 (CT) 的精度最高 (20.7%),其次是放射学 (17.3%) 和磁共振成像 (MRI) (13.9%).
- 临床背景显著提高了准确性 (10.6%至24.0%),但对文本信息的依赖性增加;整体幻觉率很高 (74.4%),因模型而异 (51.7%-82.8%).
结论:
- 目前的VLLM在放射学诊断方面显示出不可靠性,表现出显著的上下文偏差和产生错误发现的高度倾向.
- 通过特定领域的培训和严格的验证,通过实质性的改进是必不可少的,才能认为VLLMs在临床上适合医学图像解释.
相关概念视频
Uncertainty in Measurement: Accuracy and Precision
100.9K
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value.
100.9K
Language
906
Language is a unique communication system that uses words and systematic rules to organize and transmit information. Unlike other forms of communication, which may involve postures, movements, odors, or vocalizations, language relies on symbols and grammar. This makes human communication distinct from that of other species, who also communicate but do not use language in the same way humans do.
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
906
Improving Translational Accuracy
14.9K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
14.9K
Heart Failure IV: Classification and Diagnostic Evaluation
370
Heart failure can be classified in various ways, with the most common classifications based on physical activity limitations, disease progression, severity, and treatment strategies.The Functional Classification of Heart Failure divides patients into four categories based on physical activity limitation due to symptom burden.Class I: Patients in this class have cardiac disease but no physical activity limitations. Ordinary activities like walking, climbing stairs, or routine tasks do not cause...
370
Positive Symptoms Schizophrenia: Hallucinations and Delusions
557
Schizophrenia is a complex psychiatric disorder characterized by a range of symptoms that significantly impact cognition, behavior, and emotional regulation. Among these, the positive symptoms stand out as they involve the addition or exaggeration of normal mental functions, deviating markedly from typical behavior and perception. Hallucinations and delusions are prominent positive symptoms, each profoundly affecting the individual's experience of reality.
Hallucinations
Hallucinations in...
Hallucinations
Hallucinations in...
557
Irritable Bowel Syndrome II: Clinical Features and Diagnostic Evaluation
768
Irritable Bowel Syndrome II: Clinical Features and Diagnostic Evaluation
Irritable Bowel Syndrome (IBS) is classified into subtypes based on the predominant bowel habits as determined by the Bristol Stool Form Scale (BSFS). The subtypes are:
Irritable Bowel Syndrome (IBS) is classified into subtypes based on the predominant bowel habits as determined by the Bristol Stool Form Scale (BSFS). The subtypes are:
768


