大型语言模型在回答食道癌相关问题的比较性能评估:多模型评估研究
Zijie He1, Lilan Zhao1, Genglin Li1
1Department of Thoracic Surgery, Shengli Clinical Medical College of Fujian Medical University, Fujian Provincial Hospital, Fuzhou University Affiliated Provincial Hospital, Fuzhou, China.
Frontiers in digital health
|October 23, 2025
概括
大型语言模型 (LLM) 对食道癌症信息有希望,具有不同的强度. 双子座在准确度方面领先,而ChatGPT提供了更多的完整性,特别是在复杂的手术细节方面.
科学领域:
- 人工智能在医学中的应用
- 瘤学研究研究
- 医疗信息学 医疗信息学
背景情况:
- 食道癌的发病率和死亡率很高,公众越来越需要可靠的在线健康信息.
- 在线医疗信息的准确性和可理解性,特别是来自人工智能的信息,仍然令人担忧.
- 本研究探讨了在医疗信息传播中评估人工智能工具的必要性.
研究的目的:
- 系统地比较领先的大型语言模型 (LLM) 的准确性,完整性和可理解性,以回答食道癌相关的查询.
- 为临床环境和患者教育选择合适的AI工具提供指导.
- 评估不同LLM之间的性能差异,以满足特定医疗信息需求.
主要方法:
- 开发了65个食道癌症问题,涵盖了知识,手术前,手术前和手术后的方面.
- 五个LLM (ChatGPT 5,Claude Sonnet 4.0,DeepSeek-R1,Gemini 2.5 Pro,Grok-4) 使用标准化的提示进行了查询.
- 答案由五位高级临床专家在五分利克特级别上评估,并使用弗里德曼和威尔科克森签名等级测试对低分进行重新测试和统计分析.
主要成果:
- 所有评估的LLM都表现出强的表现,平均得分高于4.0.
- 双子 2.5 Pro显示出卓越的准确性,而ChatGPT 5在完整性方面表现出色,特别是在手术和术后信息方面.
- 在复杂的医疗领域发现了显著的性能差异,重复测试在质量改进方面产生了不同的结果.
结论:
- 在提供食道癌症信息方面,LLM具有显著的潜力,但其完整性差异很大.
- 在复杂的场景中,ChatGPT表现出更大的全面性,而Gemini则提供更高的准确性.
- 这些发现支持在临床实践中对人工智能工具的分层应用策略,强调对LLM限制的用户教育.
更多相关视频
03:14Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
1.0K
07:13Comparison of Predictive Performance of Three Lymph Node Staging Systems in Colorectal Signet Ring Cell Carcinoma Based on Machine Learning Model
Published on: April 18, 2025
485
相关概念视频
Mouse Models of Cancer Study
Mice have long served as models for studying human biology and pathology because of their phylogenetic and physiological similarity with humans. They are also easy to maintain and breed in the laboratory, and hence, many inbred strains are now available for research. Studies on mice have contributed immeasurably to our understanding of cancer biology.
The development of transgenic, knockout, and knock-in mice has led to an exponential increase in their use as model organisms in research,...
The development of transgenic, knockout, and knock-in mice has led to an exponential increase in their use as model organisms in research,...
Mouse Models of Cancer Study
Mice have long served as models for studying human biology and pathology because of their phylogenetic and physiological similarity with humans. They are also easy to maintain and breed in the laboratory, and hence, many inbred strains are now available for research. Studies on mice have contributed immeasurably to our understanding of cancer biology.
The development of transgenic, knockout, and knock-in mice has led to an exponential increase in their use as model organisms in research,...
The development of transgenic, knockout, and knock-in mice has led to an exponential increase in their use as model organisms in research,...
