对大型语言模型在具有挑战性的临床病例中的诊断能力进行比较
Maria Palwasha Khan1, Eoin Daniel O'Sullivan1,2
1Kidney Health Service, Metro North Hospital and Health Service, Brisbane, QLD, Australia.
Frontiers in artificial intelligence
|August 20, 2024
概括
与其他大型语言模型 (LLM) 相比,Gemini在诊断复杂的临床病例方面表现出卓越的表现. 一个新的分级工具可靠地评估了LLM诊断的准确性和安全性.
科学领域:
- 人工智能在医学中的应用
- 临床决策支持系统 临床决策支持系统
- 自然语言处理自然语言处理.
背景情况:
- 面向消费者的大型语言模型 (LLM) 越来越容易获得,具有临床诊断支持的潜力.
- 在复杂的医疗场景中评估不同LLM的性能对于安全有效的实施至关重要.
研究的目的:
- 在解决复杂的临床病例时,比较常用大型语言模型 (LLM) 的诊断性能.
- 评估一种新的标题在临床环境中对LLM输出进行分级的有用性.
主要方法:
- 对三个LLM (Bing,ChatGPT,Gemini) 进行了比较分析,使用来自新英格兰医学杂志病例系列的临床病例.
- 一个新开发的标题评估了基于准确性,可读性,临床解释性和安全性的LLM绩效.
主要成果:
- 当呈现相同的临床信息时,观察到LLM表现的显著差异.
- 双子座在评估的法学士中表现最高.
- 开发的分级标签在评估LLM临床输出方面显示了观察者间的低变化和可靠性.
结论:
- 在临床场景中,LLM的表现有很大差异,这强调了在部署之前需要仔细评估的必要性.
- 该研究引入了一种可靠的工具,用于评估LLM生成的诊断支持的临床实用性和安全性.
更多相关视频
05:56Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application
Published on: April 14, 2023
2.4K
08:05Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
Published on: June 30, 2020
7.5K
相关概念视频
01:27Language and Cognition
339
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
339
01:15Receiver Operating Characteristic Plot
106
A ROC (Receiver Operating Characteristic) plot is a graphical tool used to assess the performance of a binary classification model by illustrating the trade-off between sensitivity (true positive rate) and specificity (false positive rate). By plotting sensitivity against 1 - specificity across various threshold settings, the ROC curve shows how well the model distinguishes between classes, with a curve closer to the top-left corner indicating a more accurate model. The area under the ROC curve...
106