临床专业人员和大型语言模型的诊断准确度的比较:系统审查和元分析
Guxue Shan1, Xiaonan Chen1, Chen Wang1
1Nanjing Drum Tower Hospital Clinical College of Nanjing University of Chinese Medicine, Nanjing, China.
JMIR medical informatics
|April 25, 2025
概括
大型语言模型 (LLM) 在医学诊断方面表现有前途,其准确性因任务而异. 虽然还没有与医生匹配,但LLM可以成为有价值的医疗保健助理,如果仔细实施.
科学领域:
- 人工智能在医学中的应用
- 医疗信息学 医疗信息学
- 临床决策支持 临床决策支持
背景情况:
- 生成型人工智能 (AI) 和大型语言模型 (LLM) 在医疗保健领域显示出巨大潜力.
- 专业的LLM可以处理复杂的医学文本,分析记录,并提供诊断建议,特别是对于罕见疾病.
- 在将LLM诊断性能与人类医生进行比较时存在关键差距.
研究的目的:
- 系统地审查临床环境中LLMs的诊断准确性.
- 为未来的LLMs临床整合提供基于证据的参考资料.
- 评估LLM在各种医疗应用中的表现.
主要方法:
- 在主要数据库 (PubMed,科学网,Embase等) 进行全面的文献搜索. 从2017年到现在.
- 由两个审查员进行独立选和数据提取.
- 偏差风险和适用性评估使用预测模型偏差风险评估工具 (PROBAST).
主要成果:
- 包括30项研究,包括19个LLM和4762个案例.
- 大多数研究中都注意到偏差的高风险,主要是由于已知的病例诊断.
- 最佳的LLM初级诊断准确度在25%至97.8%之间;分拣准确度在66.5%至98%之间.
结论:
- 法律学具有显著的诊断能力和广泛的临床应用潜力.
- 尽管与医生相比,目前的准确性有限,但谨慎使用可以将LLM定位为有效的医疗保健助理.
- 建议进一步的研究和仔细的实施将LLMs纳入临床实践.
相关概念视频
Improving Translational Accuracy
2.5K
2.5K
Sensitivity, Specificity, and Predicted Value
135
In healthcare diagnostics, laboratory tests play a crucial role in identifying and diagnosing a wide range of medical conditions. However, interpreting test results is not always straightforward. An abnormal test result does not always confirm the presence of a disease, just as a normal result does not guarantee its absence. To assess the reliability of these diagnostic tools, healthcare practitioners rely on two key statistical indicators: sensitivity and specificity.
Sensitivity is the...
Sensitivity is the...
135
Receiver Operating Characteristic Plot
64
A ROC (Receiver Operating Characteristic) plot is a graphical tool used to assess the performance of a binary classification model by illustrating the trade-off between sensitivity (true positive rate) and specificity (false positive rate). By plotting sensitivity against 1 - specificity across various threshold settings, the ROC curve shows how well the model distinguishes between classes, with a curve closer to the top-left corner indicating a more accurate model. The area under the ROC curve...
64
Language and Cognition
291
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
291


