评估中等规模的开源大型语言模型:在精确瘤护理环境中提供决策支持
Kevin Kaufmes1, Georg Mathes1, Dilyana Vladimirova2
1MOLIT Institute, Heilbronn, Germany.
Studies in health technology and informatics
|September 3, 2025
概括
中等规模的大型语言模型 (LLM) 对精密瘤分子瘤板 (MTB) 准备的可靠性不足. 目前的LLM经常提供过时或不正确的信息,对患者的安全构成风险.
科学领域:
- 医学的人工智能
- 临床决策支持系统
背景情况:
- 精确瘤学需要对复杂病例的最新知识.
- 准备用于分子瘤板 (MTB) 的病例是劳动密集的.
- 大型语言模型 (LLM) 有助于简化MTB的信息检索.
研究的目的:
- 评估中等规模的LLM在MTB准备中回答临床问题的有用性.
- 评估使用消费者硬件处理敏感数据的本地LLM的性能.
主要方法:
- 根据基准和推理能力选择了三个LLM.
- 该领域的专家提供了与MTB相关的示例问题.
- 专家对LLM产生的答案进行了质量和正确性的评估.
主要成果:
- 总体而言,LLM的表现不佳,并发现了一些重大问题.
- 很大一部分回复包含过时,不完整或事实上错误的信息.
- 观察到评估器的不一致性和不同的置信度.
结论:
- 目前中等规模的LLM在精密瘤应用中是不可靠的.
- 过时的信息和充满信心的错误信息凸显了基准和现实世界表现之间的差距.
- 未来的研究应该探索RAG和网络搜索等先进技术,优先考虑患者的安全.
相关概念视频
Combination Therapies and Personalized Medicine
5.1K
Combining two or more treatment methods increases the life span of cancer patients while reducing damage to vital organs or tissue from the overuse of a single treatment. Combination therapy also targets different cancer-inducing pathways, thus reducing the chances of developing resistance to treatment.
The combination of the drug acetazolamide and sulforaphane is a good example of combination therapy to treat cancer. The cells in the interior of a large tumor often die due to the hypoxic and...
The combination of the drug acetazolamide and sulforaphane is a good example of combination therapy to treat cancer. The cells in the interior of a large tumor often die due to the hypoxic and...
5.1K
Cancer Survival Analysis
448
Cancer survival analysis focuses on quantifying and interpreting the time from a key starting point, such as diagnosis or the initiation of treatment, to a specific endpoint, such as remission or death. This analysis provides critical insights into treatment effectiveness and factors that influence patient outcomes, helping to shape clinical decisions and guide prognostic evaluations. A cornerstone of oncology research, survival analysis tackles the challenges of skewed, non-normally...
448


