医学大语言模型在回答现实世界辐射瘤学问题的比较评估:多中心观测研究
Fabio Dennstädt1, Max Schmerder1, Elena Riggenbach1
1Inselspital, Department of Radiation Oncology, Bern University Hospital, University of Bern, Bern, Switzerland.
Journal of medical Internet research
|September 23, 2025
概括
一个本地部署的大型语言模型 (LLM) 在回答辐射瘤学问题时表现出与临床专家相似的性能. 这种医疗人工智能显示出临床支持的潜力,尽管需要进一步评估才能得到广泛采用.
科学领域:
- 人工智能在医学中的应用
- 临床决策支持系统 临床决策支持系统
- 辐射瘤学应用 辐射瘤学应用
背景情况:
- 大型语言模型 (LLM) 在辐射瘤学等专业领域的临床任务中表现有前途.
- 之前的LLM评估专注于考试设置,让现实世界的临床表现不确定.
- 当地部署的LLM作为临床问题的AI助理的潜力仍有待确定.
研究的目的:
- 评估一家最先进的医学LLM与临床专家在回答现实世界的放射瘤学问题上的表现.
- 评估LLM产生的临床决策答案的质量和潜在危害性.
主要方法:
- 医生们收集了来自10家欧洲医院的临床放射瘤学问题.
- 三位高级放射瘤学专家和LLM Llama3-OpenBioLLM-70B.回答了50个问题.
- 医生进行了盲目审查,评价了答案质量,潜在的有害性和来源的可识别性 (专家与LLM).
主要成果:
- 在LLM和临床专家之间没有发现答案质量的显著差异 (平均得分为3.38比3.63).
- 潜在有害的答案发生在16%的LLM答案中,而临床专家的答案为13% (不具有统计学意义).
- 医生正确地确定了答案的来源78%的时间专家和72%的LLM.
结论:
- 当地部署的医学LLM在回答有关质量和安全的放射瘤学问题时,与临床专家相提并论.
- 虽然LLM在医院部署方面变得越来越有能力和负担得起,但尚未准备好作为一般助理在临床上全面实施.
- 现实世界的评估研究对于了解LLM的局限性和指导负责任的临床整合至关重要,需要对生成AI进行医疗保健专业教育.
更多相关视频
07:15Machine Learning Algorithms for Early Detection of Bone Metastases in an Experimental Rat Model
Published on: August 16, 2020
7.4K
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
1.3K
相关概念视频
01:21Cancer Survival Analysis
650
Cancer survival analysis focuses on quantifying and interpreting the time from a key starting point, such as diagnosis or the initiation of treatment, to a specific endpoint, such as remission or death. This analysis provides critical insights into treatment effectiveness and factors that influence patient outcomes, helping to shape clinical decisions and guide prognostic evaluations. A cornerstone of oncology research, survival analysis tackles the challenges of skewed, non-normally...
650
01:29Positron Emission Tomography
7.0K
Positron emission tomography (PET) is a medical imaging technique involving radiopharmaceuticals — substances that emit short-lived radiation. Although the first PET scanner was introduced in 1961, it took 15 more years before radiopharmaceuticals were combined with the technique and revolutionized its potential.
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body...
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body...
7.0K
01:20Comparing the Survival Analysis of Two or More Groups
561
Survival analysis is a cornerstone of medical research, used to evaluate the time until an event of interest occurs, such as death, disease recurrence, or recovery. Unlike standard statistical methods, survival analysis is particularly adept at handling censored data—instances where the event has not occurred for some participants by the end of the study or remains unobserved. To address these unique challenges, specialized techniques like the Kaplan-Meier estimator, log-rank test, and...
561