赋予蛋白质语言模型结构知识
Philip Hartout1, Dexiong Chen1, Paolo Pellizzoni1
1Department of Machine Learning and Systems Biology, Max Planck Institute of Biochemistry, Bavaria, Martinsried 82152, Germany.
Bioinformatics (Oxford, England)
|October 29, 2025
概括
我们开发了一种计算效率高的方法,将蛋白质序列和结构数据结合起来,以改善蛋白质表示学习. 这种方法可以提高性能,而不需要现有的结构感知模型的高计算成本.
科学领域:
- 计算生物学 计算生物学
- 结构生物信息学 结构生物信息学
- 机器学习用于蛋白质.
背景情况:
- 蛋白质语言模型 (PLM) 擅长从序列数据中学习,但往往忽视结构信息.
- 将结构数据集成到PLM中是计算密集和复杂的.
- 由于高资源需求,现有的方法限制了实际采用.
研究的目的:
- 为蛋白质开发一个计算高效的关节序列和结构嵌入方法.
- 为了使结构知识无地纳入现有的PLM.
- 为了平衡高性能与实际的计算约束.
主要方法:
- 一个新的轻量级集成框架,将预训练的序列变压器与专门的结构适配器相结合.
- 增强自我注意力机制,以纳入结构知识.
- 使用面具语言建模,对542K蛋白质结构进行适度的预训练.
主要成果:
- 实现了计算和参数效率.
- 它的性能优于ESM-2等仅序列模型.
- 基于结构的复杂方法的性能与显著更少的资源相匹配.
- 通过适度的预训练数据证明了效率.
结论:
- 建立了蛋白质表示学习的新范式.
- 为捕获序列和结构蛋白质信息提供了一个可访问的工具.
- 克服了通常与结构意识模型相关的计算开销.
相关概念视频
01:22Language Development
831
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
831
01:24Components of Language
746
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs.
746
01:27Language and Cognition
704
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
704
01:56Structural Protein Function
3.2K
3.2K
01:56Structural Protein Function
29.7K
Structural proteins are a category of proteins responsible for functions ranging from cell shape and movement to providing support to major structures such as bones, cartilage, hair, and muscles. This group includes proteins such as collagen, actin, myosin, and keratin.
Collagen, the most abundant protein in mammals, is found throughout the body. In connective tissue, such as skin, ligaments, and tendons, it provides tensile strength and elasticity. In bones and teeth, it mineralizes to...
Collagen, the most abundant protein in mammals, is found throughout the body. In connective tissue, such as skin, ligaments, and tendons, it provides tensile strength and elasticity. In bones and teeth, it mineralizes to...
29.7K
01:10Higher Mental Functions of the Brain: Language
3.4K
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
3.4K