从大语言中转移学习与临床概念嵌入
Yuhe Gao1, Runxue Bao2, Yuelyu Ji1
1University of Pittsburgh.
概括
域特定的大型语言模型 (LLM) 通过捕捉临床概念语义来改善跨站点的医疗保健知识传输. 微调通用LLM对于跨站点数据分析的最佳性能至关重要.
科学领域:
- 生物医学信息学 生物医学信息学
- 医疗保健中的人工智能
背景情况:
- 医疗保健领域的知识交流对于数据稀缺和及时干预至关重要.
- 跨站点的临床概念异质性对跨站点的知识转移构成了挑战.
- 大型语言模型 (LLM) 在理解临床语义和减少异质性方面表现有前途.
研究的目的:
- 评估LLM生成的语义嵌入对医疗保健数据的影响.
- 使用LLM嵌入的本地,共享和转移学习模型的性能进行比较.
- 评估领域特定与通用LLM在跨站点知识转移中的有效性.
主要方法:
- 分析来自两个大型医疗保健系统的电子健康记录.
- 使用源自域特定 (Med-BERT) 和通用 (OpenAI) LLMs 的语义嵌入.
- 在本地,共享和转移学习环境中评估模型性能.
主要成果:
- 特定领域的LLM (Med-BERT) 在本地和直接转移学习场景中表现出卓越的表现.
- 一般的LLM (OpenAI嵌入) 可能需要微调,以在跨站点应用中获得最佳结果.
- LLM嵌入式显著影响了各种建模方法的性能.
结论:
- 特定领域的LLM嵌入对于有效的医疗保健知识转移至关重要.
- 精细的模型调整对于在异质临床数据中利用LLM至关重要.
- 未来的研究应该探索任务复杂性,数据大小和在医疗保健中LLM应用的模型调整之间的相互作用.
更多相关视频
12:49Transcranial Direct Current Stimulation tDCS of Wernicke's and Broca's Areas in Studies of Language Learning and Word Acquisition
Published on: July 13, 2019
16.8K
10:15Utilizing Repetitive Transcranial Magnetic Stimulation to Improve Language Function in Stroke Patients with Chronic Non-fluent Aphasia
Published on: July 2, 2013
17.8K
相关概念视频
Concepts and Prototypes
119
The human nervous system handles vast amounts of information by translating sensory stimuli into neural impulses, which the brain processes, creating thoughts expressed through language or stored as memories. The brain also synthesizes information from emotions and memories, which significantly influence thoughts and behaviors. This intricate process creates a comprehensive mental picture.
The brain organizes this information using concepts, which are mental categories grouping linguistic data,...
The brain organizes this information using concepts, which are mental categories grouping linguistic data,...
119
Language and Cognition
336
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
336
Long-term Potentiation
54.9K
Long-term potentiation, or LTP, is one of the ways by which synaptic plasticity—changes in the strength of chemical synapses—can occur in the brain. LTP is the process of synaptic strengthening that occurs over time between pre- and postsynaptic neuronal connections. The synaptic strengthening of LTP works in opposition to the synaptic weakening of long-term depression (LTD) and together are the main mechanisms that underlie learning and memory.
54.9K
Elaborative Rehearsals
83
Elaborative rehearsal is a crucial cognitive strategy that strengthens information encoding in long-term memory by making meaningful connections between new data and pre-existing knowledge. This approach contrasts with maintenance rehearsal, which involves simple repetition without delving into the significance of the information. While maintenance rehearsal might temporarily keep information active in short-term memory, it is less effective for long-term retention.
The effectiveness of...
The effectiveness of...
83
Clinical Trials
6.6K
Clinical trials are prospective experimental studies conducted on humans to determine the safety and efficacy of treatments, drugs, diet methods, and medical devices. Using statistics in clinical trials enables researchers to derive reasonable and accurate conclusions from the collected data, allowing them to make wise decisions in uncertain situations. In medical research, statistical methods are crucial for preventing errors and bias.
There are four phases in a clinical trial. A phase one...
There are four phases in a clinical trial. A phase one...
6.6K
Clinical Trials: Overview
2.9K
Clinical development focuses on how the drug will interact with the human body and encompasses four key phases of clinical trials, each serving a specific purpose in assessing the safety and effectiveness of new drugs. These phases overlap and build upon one another. Phase I involves a small group of healthy volunteers (typically 20-80 individuals) or, in cases where significant toxicity is expected, patients with the targeted disease, such as cancer or AIDS. The volunteers are tested for...
2.9K
