通过子词优化和嵌入初始化,将临床知识注入语言模型中
Abul Hasan1, Jinge Wu1, Quang Ngoc Nguyen1
1University College London, Institute of Health Informatics, 222 Euston Rd., London, NW1 2DA, UK.
Computers in biology and medicine
|August 8, 2025
概括
一种新方法,K-Tokeniser,通过结合医学知识来增强临床语言模型. 这种方法可以提高各种任务的性能,并加快模型培训,而不需要预先培训.
科学领域:
- 自然语言处理 (NLP) 是一种自然语言处理.
- 医疗信息学 医疗信息学
- 机器学习 机器学习
背景情况:
- 临床文本处理需要复杂的语言模型.
- 现有的代币化方法可能无法完全捕捉临床领域的知识.
- 将医学语义纳入语言模型对于准确性至关重要.
研究的目的:
- 介绍K-Tokeniser,这是一个用于临床文本的新型代币化方法.
- 将临床知识注入语言模型,以提高性能.
- 在临床自然语言处理任务中增强语义理解.
主要方法:
- K-Tokeniser使用域本体学 (例如,UMLS) 或语料库数据来填充令牌表示.
- 它利用句子级上下文在训练/推理过程中选择最佳的全球代码表示.
- 嵌入初始化方法支持新的代币,没有预先训练.
主要成果:
- 在基于变压器的模型和四个现实世界的临床数据集中,K-Tokeniser展示了一致的改进.
- 在自动化临床编码 (13%的Micro F1得分增加) 中观察到显著的收益.
- K-Tokeniser促进了语言模型的更快的融合,减少了数据需求.
结论:
- 使用K-Tokeniser的模型表现出更快的融合和更好的性能.
- 在训练数据显著减少的情况下实现了基线性能 (例如,自动编码的20%).
- 可泛化方法不需要预先培训,提高了其适用性.
更多相关视频
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
578
12:49Transcranial Direct Current Stimulation tDCS of Wernicke's and Broca's Areas in Studies of Language Learning and Word Acquisition
Published on: July 13, 2019
17.2K
相关概念视频
Improving Translational Accuracy
11.9K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
11.9K
Language Development
449
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
449
Components of Language
392
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs.
392
Higher Mental Functions of the Brain: Language
1.0K
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
1.0K
Language and Cognition
441
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
441
Clinical Trials
8.3K
Clinical trials are prospective experimental studies conducted on humans to determine the safety and efficacy of treatments, drugs, diet methods, and medical devices. Using statistics in clinical trials enables researchers to derive reasonable and accurate conclusions from the collected data, allowing them to make wise decisions in uncertain situations. In medical research, statistical methods are crucial for preventing errors and bias.
There are four phases in a clinical trial. A phase one...
There are four phases in a clinical trial. A phase one...
8.3K
