提升对老年人认知能力下降的早期检测:在临床笔记中使用大型语言模型进行比较研究
Xinsong Du1, John Novoa-Laurentiev2, Joseph M Plasek1
1Division of General Internal Medicine and Primary Care, Brigham and Women's Hospital, Boston, MA, 02115, USA; Department of Medicine, Harvard Medical School, Boston, MA, 02115, USA.
EBioMedicine
|October 13, 2024
概括
结合大型语言模型 (LLM) 和传统机器学习的整体模型显著改善了电子健康记录 (EHR) 中认知衰退迹象的检测. 这种方法通过利用互补的错误档案来提高诊断准确性.
科学领域:
- 医疗保健中的人工智能
- 临床信息学 临床信息学
- 自然语言处理自然语言处理.
背景情况:
- 大型语言模型 (LLM) 在医疗保健中表现有前途,但需要评估以检测电子健康记录 (EHR) 中的特定临床条件.
- 本研究评估了在临床笔记中识别认知衰退指标的LLM,并将其性能和错误模式与传统模型进行比较.
- 了解这些差异可以为改善临床环境中LLM绩效的策略提供信息.
研究的目的:
- 评估大型语言模型 (LLM) 在从电子健康记录 (EHR) 临床笔记中检测认知衰退方面的有效性.
- 将LLM (GPT-4,Llama 2) 的错误概况与传统的机器学习模型进行比较.
- 开发一种基于LLM的最佳方法来识别认知衰退,并评估集合模型的性能.
主要方法:
- 从大众医院布里格姆医院 (2019年轻度认知障碍诊断) 的临床笔记分析.
- 使用各种方法 (硬提示,检索增强生成) 开发和比较GPT-4和Llama 2的提示.
- 创建一个组合模型,将LLMs和传统模型 (层次关注网络,XGBoost) 结合起来,使用多数投票,以基于混矩阵的分数进行评估.
主要成果:
- 与Llama 2相比,GPT-4显示出更高的精度和效率,但在单独方面并没有超越传统模型.
- 整体模型在所有指标上显著优于单个模型 (p < 0.01),达到90.2%的精度,94.2%的回忆率和92.1%的F1分数.
- 整体模型显著提高了精度 (从70%-79%到>90%),所有模型的相互错误仅为3.2%,表明不同的错误配置文件.
结论:
- 在本地EHR数据上训练的LLM和传统模型具有不同的错误配置文件.
- 结合这些模型的整体方法是互补的,并提高认知衰退检测的诊断性能.
- 未来的研究应该探索将LLM与本地化模型和特定领域的知识集成在一起,以提高特定任务的性能.
更多相关视频
相关概念视频
01:27Language and Cognition
332
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
332
01:30Cognitive Development During Adulthood
65
Cognitive development continues throughout adulthood, undergoing significant shifts across early, middle, and late stages. Individual transition occurs from adolescent idealism to pragmatic and adaptable thinking in early adulthood. During this period, individuals learn to integrate personal beliefs with the recognition that other perspectives are equally valid. Exposure to the complexities of modern society, diverse experiences, and higher education contribute to this adaptive thought process,...
65
01:30Dementia
97
Dementia is a collective term for cognitive disorders primarily affecting memory, thinking, and reasoning. It is not a specific disease but a syndrome, with Alzheimer's disease being the most common cause, accounting for approximately 60-80% of cases. Other types include vascular dementia, Lewy body dementia, and frontotemporal dementia. Dementia affects millions worldwide, particularly older adults, though it is not a normal part of aging.
The progression of dementia is generally gradual....
The progression of dementia is generally gradual....
97
01:30Cognitive Enhancers: Cholinesterase Inhibitors and NMDA Receptor Antagonists
106
Cognitive enhancers, also known as "smart drugs," are substances used to enhance memory, mental alertness, and concentration. These can be natural or synthetic and improve cognition in conditions like Alzheimer's disease (AD) and other neurodegenerative diseases. Some common examples include caffeine, amphetamines, methylphenidate, modafinil, arecoline, donepezil, vortioxetine, and piracetam. These enhancers work on the principle of synaptic plasticity and altered circuit function.
106
01:26Alzheimer's Disease: Overview
449
Alzheimer's Disease (AD) is a continually advancing neurodegenerative disorder, distinguished by escalating memory loss, cognitive dysfunction, and dementia. The disease unfolds in three stages: preclinical, mild cognitive impairment (MCI), and dementia. Its onset is insidious, and the progression gradual, with the cause not well explained by other disorders.
The clinical diagnosis of AD hinges on the presence of memory and other cognitive impairments. Biomarkers, such as changes in Aβ...
The clinical diagnosis of AD hinges on the presence of memory and other cognitive impairments. Biomarkers, such as changes in Aβ...
449