对胃肠病学临床推理的专有和开源语言和视觉语言模型进行比较
Seyed Amir Ahmad Safavi-Naini1,2,3, Shuhaib Ali4, Omer Shahab5
1Division of Data-Driven and Digital Health (D3M), The Charles Bronfman Institute for Personalized Medicine, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
NPJ digital medicine
|November 27, 2025
概括
大型语言模型 (LLM) 和视觉语言模型 (VLM) 在胃肠病学中表现有前途. 像o1-preview和Claude 3.5 Sonnet这样的专有模型的表现优于开源选项,而量子化模型的表现与完全精确的模型相比.
科学领域:
- 医疗人工智能 医疗人工智能
- 胃肠病学应用 胃肠病学应用
- 自然语言处理自然语言处理.
背景情况:
- 先进AI的整合,包括大语言模型 (LLM) 和视觉语言模型 (VLM),正在跨医学学科迅速扩展.
- 在胃肠病学等专业领域评估这些模型的性能对于理解它们的潜在临床实用性至关重要.
研究的目的:
- 在胃肠病学背景下评估各种专有和开源LLM和VLM的有效性和准确性.
- 在不同的接口,计算环境和量子化水平上比较模型性能.
主要方法:
- 使用板式的多选题,与胃肠道学相关,用于测试LLM和VLM (例如,GPT,Claude,Gemini,Mistral,Llama,Mixtral,Phi,Qwen).
- 在不同的条件下评估性能,包括开源模型的不同压缩 (量子化) 级别.
- 研究了图像标题 (人为生成的,原始的,LLM生成的) 对基于图像的问题VLM性能的影响.
主要成果:
- 专有模型o1-preview (82.0%) 和Claude3.5-Sonnet (74.0%) 获得了最高的准确性.
- 顶级开源模型Llama3.3-70b (65.7%) 和Qwen-2.5-72b (61.0%) 的表现优于专有模型.
- 小型的量子化开源模型 (8位Llama 3.2-11b,6位Phi3-14b) 的性能与它们的全精度版本相当.
- 视觉语言模型的准确性增加了人类生成的标题,但在基于图像的问题上,LLM生成的标题下降了.
结论:
- 与当前的开源模型相比,专有LLM和VLM在胃肠病学评估中表现优越.
- 量子化提供了一种可行的方法,可以减少开源模型的计算要求,而不会造成显著的准确性损失.
- 图像标题的质量极大地影响了VLM的性能,强调了临床应用中需要仔细准备数据的必要性.
相关概念视频
Imaging Studies III: Gastrointestinal Motility Studies and Virtual Colonoscopy
353
This lesson explores three gastrointestinal imaging techniques: radionuclide testing, colonic transit studies, and virtual colonoscopy.
Radionuclide Testing
Radionuclide testing is a sophisticated medical technique for assessing gastrointestinal motility. It focuses on gastric emptying and colonic transit time. Radioactive markers track the movement of food through the digestive system, providing insights into gastrointestinal disorders.
In gastric emptying studies, a meal's liquid and...
Radionuclide Testing
Radionuclide testing is a sophisticated medical technique for assessing gastrointestinal motility. It focuses on gastric emptying and colonic transit time. Radioactive markers track the movement of food through the digestive system, providing insights into gastrointestinal disorders.
In gastric emptying studies, a meal's liquid and...
353
Assessment of the Gastrointestinal System II: Health Perception Pattern
434
Assessing the gastrointestinal (GI) system is a complex process that begins with collecting subjective data. This data, collected through patient interviews, provides crucial insights into the patient's health history, perception patterns, and lifestyle habits, all contributing significantly to GI health.
Health Perception Patterns
Health perception patterns offer valuable insights into a patient's lifestyle habits and how they may impact their GI health. These patterns include:
Health Perception Patterns
Health perception patterns offer valuable insights into a patient's lifestyle habits and how they may impact their GI health. These patterns include:
434
Pharmacokinetic Models: Comparison and Selection Criterion
309
Physiological and compartmental models are valuable tools used in studying biological systems. These models rely on differential equations to maintain mass balance within the system, ensuring an accurate representation of the dynamic processes at play.
Physiological models take a detailed approach by considering specific molecular processes. They can predict drug distribution, metabolism, and elimination changes, providing a comprehensive understanding of how drugs interact with the body.
Physiological models take a detailed approach by considering specific molecular processes. They can predict drug distribution, metabolism, and elimination changes, providing a comprehensive understanding of how drugs interact with the body.
309
Language and Cognition
693
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
693


