总结使用癌症治疗的大型语言模型的临床证据:一个盲目的比较分析
Samuel Rubinstein1, Aleenah Mohsin2, Rahul Banerjee3,4
1Division of Hematology, Department of Medicine, University of North Carolina, Chapel Hill, NC, United States.
Frontiers in digital health
|May 14, 2025
概括
与其他大型语言模型 (LLM) 相比,Claude 3.5 在生成临床证据总结方面表现出卓越的表现,尽管所有模型都需要专家编辑. 需要进一步的研究来评估LLM适合独立的临床总结生成.
科学领域:
- 人工智能在医学中的应用
- 临床证据综合研究
- 血液学 血液学 血液学
背景情况:
- 简洁的临床证据总结对于治疗决策至关重要,但需要大量的劳动来创建.
- 大型语言模型 (LLM) 为高效的总结生成提供了一个潜在的解决方案.
- 然而,LLM可能会产生不准确的信息,需要客观评估.
研究的目的:
- 客观地评估四种商业大型语言模型 (LLM) 在生成临床证据总结方面的能力.
- 为了比较Claude 3.5,ChatGPT 4.0,Gemini 1.0和Llama-3.1在创建多发性骨髓瘤和AL Amyloidosis治疗方案中的突触的性能.
主要方法:
- 四名LLM (Claude 3.5,ChatGPT 4.0,Gemini 1.0,Llama-3.1) 被要求为六种治疗方案生成总结.
- 两位血液学家独立评估了根据利克特尺度的准确性,完整性,相关性,清晰性,连贯性和幻觉的总结.
- 绩效指标包括平均利克特分数和通过科恩的二次加权卡帕通过评分机构之间的可靠性.
主要成果:
- 克劳德3.5在所有评估领域中获得了最高分,特别是在准确性 (3.92) 和完整性 (4.00) 中.
- 聊天GPT 4.0和Gemini 1.0表现中等,而Llama-3.1在所有指标上表现明显低.
- 没有LLM在准确性,完整性或相关性方面取得了完美的成绩,这表明需要专家审查.
结论:
- 克劳德3.5在产生临床总结方面始终优于其他LLM.
- 所有经过测试的LLM都需要领域专家仔细编辑以确保可用性.
- 在独立生成临床总结的LLMs的适用性需要进一步的研究.
相关概念视频
Cancer Survival Analysis
311
Cancer survival analysis focuses on quantifying and interpreting the time from a key starting point, such as diagnosis or the initiation of treatment, to a specific endpoint, such as remission or death. This analysis provides critical insights into treatment effectiveness and factors that influence patient outcomes, helping to shape clinical decisions and guide prognostic evaluations. A cornerstone of oncology research, survival analysis tackles the challenges of skewed, non-normally...
311
Combination Therapies and Personalized Medicine
4.8K
Combining two or more treatment methods increases the life span of cancer patients while reducing damage to vital organs or tissue from the overuse of a single treatment. Combination therapy also targets different cancer-inducing pathways, thus reducing the chances of developing resistance to treatment.
The combination of the drug acetazolamide and sulforaphane is a good example of combination therapy to treat cancer. The cells in the interior of a large tumor often die due to the hypoxic and...
The combination of the drug acetazolamide and sulforaphane is a good example of combination therapy to treat cancer. The cells in the interior of a large tumor often die due to the hypoxic and...
4.8K
Targeted Cancer Therapies
7.4K
The targeted cancer therapies, also known as “molecular targeted therapies,” take advantage of the molecular and genetic differences between the cancer cells and the normal cells. It needs a thorough understanding of the cancer cells to develop drugs that can target specific molecular aspects that drive the growth, progression, and spread of cancer cells without affecting the growth and survival of other normal cells in the body.
There are several types of targeted therapies against...
There are several types of targeted therapies against...
7.4K
Hazard Ratio
72
The hazard ratio (HR) is a widely used measure in clinical trials to compare the risk of events, such as death or disease recurrence, between two groups over time. It reflects the ratio of hazard rates—the instantaneous risk of the event occurring—between a treatment group and a control group. This measure provides valuable insights into the relative effectiveness of a treatment by assessing how the risk of an event differs between the two groups.
For example, in a clinical trial...
For example, in a clinical trial...
72
Mouse Models of Cancer Study
5.5K
Mice have long served as models for studying human biology and pathology because of their phylogenetic and physiological similarity with humans. They are also easy to maintain and breed in the laboratory, and hence, many inbred strains are now available for research. Studies on mice have contributed immeasurably to our understanding of cancer biology.
The development of transgenic, knockout, and knock-in mice has led to an exponential increase in their use as model organisms in research,...
The development of transgenic, knockout, and knock-in mice has led to an exponential increase in their use as model organisms in research,...
5.5K
Cancer Therapies
7.5K
Cancer therapies are various modes of treatment, such as surgery, radiation therapy, and chemotherapy that are administered to cancer patients.
However, cancer treatments can pose several challenges, as therapies used to kill cancer cells are generally also toxic to normal cells. Moreover, cancer cells mutate rapidly and can develop resistance to chemical agents or radiation therapy. Besides, all types of cancer cells may not respond to the same therapy. Some cancer cells respond to one...
However, cancer treatments can pose several challenges, as therapies used to kill cancer cells are generally also toxic to normal cells. Moreover, cancer cells mutate rapidly and can develop resistance to chemical agents or radiation therapy. Besides, all types of cancer cells may not respond to the same therapy. Some cancer cells respond to one...
7.5K


