在瘤学中基于证据的治疗建议的AI:对大型语言模型和代理工作流程的盲目评估
Guannan Zhai1, Merav Bar2, Andrew J Cowan3,4
1Department of Statistics, George Washington University, Washington, DC, United States.
Frontiers in artificial intelligence
|December 25, 2025
概括
大型语言模型 (LLM) 显示出基于证据的医学的潜力,但需要临床监督. 在多发性骨髓瘤护理的准确性和相关性方面,代理工作流系统的表现优于一般的LLM.
科学领域:
- 人工智能在医学中的应用
- 瘤学的决策支持
背景情况:
- 基于证据的医学对临床决策至关重要.
- 当前的治疗决策往往缺乏最新证据的整合.
- 大型语言模型 (LLM) 具有弥合这一差距的潜力,但其可靠性尚未得到证实.
研究的目的:
- 评估和比较五个大型语言模型 (LLM) 在为多发性骨髓瘤提供医疗建议方面的表现.
- 在准确性,相关性,全面性,幻觉率和临床使用准备等维度中评估LLM.
主要方法:
- 在50个多发性骨髓瘤临床场景中比较了五个LLM (三个通用,一个检索增强生成 (RAG),一个代理工作流).
- 利用了2024年6月的统一证据截止日期.
- 通过三个独立的血液学家瘤学家使用标准化的标准来评估反应.
主要成果:
- 代理工作流系统 (HopeAI) 在准确度 (82.0%),相关性 (85.3%) 和全面性 (74.0%) 中获得了最高分.
- 一般目的的LLM (OpenAI o1-预览,克劳德 3.5 索内特,双子 1.5 Pro) 和RAG系统 (Myelo) 在这些指标中表现较差.
- 所有系统的幻觉率都很低 (HopeAI:5.3%).
- 在所有评估的LLM中,临床准备度得分很低.
结论:
- 目前的LLM在医疗决策支持方面表现有前途,但需要仔细的临床监督.
- 需要进一步的研究,以提高LLM临床使用准备在瘤学工作流程.
- 研究结果为LLM在瘤学方面的能力和局限性提供了洞察力,指导未来AI在临床实践中的整合.
相关概念视频
Cancer Survival Analysis
626
Cancer survival analysis focuses on quantifying and interpreting the time from a key starting point, such as diagnosis or the initiation of treatment, to a specific endpoint, such as remission or death. This analysis provides critical insights into treatment effectiveness and factors that influence patient outcomes, helping to shape clinical decisions and guide prognostic evaluations. A cornerstone of oncology research, survival analysis tackles the challenges of skewed, non-normally...
626
Combination Therapies and Personalized Medicine
5.8K
Combining two or more treatment methods increases the life span of cancer patients while reducing damage to vital organs or tissue from the overuse of a single treatment. Combination therapy also targets different cancer-inducing pathways, thus reducing the chances of developing resistance to treatment.
The combination of the drug acetazolamide and sulforaphane is a good example of combination therapy to treat cancer. The cells in the interior of a large tumor often die due to the hypoxic and...
The combination of the drug acetazolamide and sulforaphane is a good example of combination therapy to treat cancer. The cells in the interior of a large tumor often die due to the hypoxic and...
5.8K
Treatment Resistant Cancers
3.7K
Cancer is the second leading cause of death in the United States. A cancer cell is genetically unstable and hence can mutate faster. They can also modify their microenvironment and escape immune surveillance. The difficulties in treating cancer are further compounded by the emergence of rapid resistance to anticancer drugs. The most common ways to attain resistance in cancer cells include alteration in drug transport and metabolism, modification of drug target, elevated DNA damage response, or...
3.7K
Targeted Cancer Therapies
8.6K
The targeted cancer therapies, also known as “molecular targeted therapies,” take advantage of the molecular and genetic differences between the cancer cells and the normal cells. It needs a thorough understanding of the cancer cells to develop drugs that can target specific molecular aspects that drive the growth, progression, and spread of cancer cells without affecting the growth and survival of other normal cells in the body.
There are several types of targeted therapies against...
There are several types of targeted therapies against...
8.6K
Cancer Therapies
9.8K
Cancer therapies are various modes of treatment, such as surgery, radiation therapy, and chemotherapy that are administered to cancer patients.
However, cancer treatments can pose several challenges, as therapies used to kill cancer cells are generally also toxic to normal cells. Moreover, cancer cells mutate rapidly and can develop resistance to chemical agents or radiation therapy. Besides, all types of cancer cells may not respond to the same therapy. Some cancer cells respond to one...
However, cancer treatments can pose several challenges, as therapies used to kill cancer cells are generally also toxic to normal cells. Moreover, cancer cells mutate rapidly and can develop resistance to chemical agents or radiation therapy. Besides, all types of cancer cells may not respond to the same therapy. Some cancer cells respond to one...
9.8K


