将实体级知识纳入预训练的语言模型,用于生物医学密集检索.
Jiajie Tan1, Jinlong Hu1, Shoubin Dong1
1Guangdong Key Lab of Computer Network, School of Computer Science and Engineering, South China University of Technology, Guangzhou, China.
Computers in biology and medicine
|October 3, 2023
概括
本研究介绍了ELK,这是一种通过整合外部知识图嵌入来增强生物医学密集检索的新方法. ELK提高了搜索结果的排名,并保持了高效的查询处理.
科学领域:
- 生物医学自然语言处理 (NLP)
- 信息检索 信息检索
- 机器学习 机器学习
背景情况:
- 预训练语言模型 (PLM) 在NLP中表现出色,但在密集检索中与生物医学实体模糊性作斗争.
- 现有的基于PLM的密集检索方法需要在多样化的生物医学领域进行改进.
研究的目的:
- 通过弥合语义差距,提高生物医学领域的密集检索性能.
- 用实体层面的外部知识来丰富查询和文档表示.
主要方法:
- 将企业级外部知识纳入基于BERT的密集检索模型.
- 引入了额外的自我注意和信息交互模块,用于融合文本和实体嵌入.
- 开发了实体相似性损失和权重实体连锁机制.
主要成果:
- 拟议的方法 (ELK) 在两个生物医学数据集上显著优于最先进的密集检索技术.
- 与coCondenser相比,ELK在NDCG指标中实现了至少5%的改进.
- 尽管其架构得到了改进,但ELK表现出了竞争性的查询延迟.
结论:
- 整合外部知识有效地解决了生物医学密集检索中的语义差距.
- 对于改善生物医学信息检索,ELK提供了一种强大而高效的解决方案.
- 该方法对在专业科学领域推进NLP应用有前途.
相关概念视频
02:07Improving Translational Accuracy
11.5K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
11.5K
01:45ER Retrieval Pathway
3.9K
In the secretory pathway, vesicles transport proteins from one cellular compartment to another in forward transport to deliver the protein to its correct location. Occasionally, misfolded proteins and incorrect proteins escape their original compartments, and a retrieval pathway is used to return the escaped proteins to their original compartment.
The ER uses many checkpoints to prevent the entry of incorrectly folded or a resident protein as cargo onto a transport vesicle. These mechanisms...
The ER uses many checkpoints to prevent the entry of incorrectly folded or a resident protein as cargo onto a transport vesicle. These mechanisms...
3.9K
01:10Higher Mental Functions of the Brain: Language
903
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
903