局部意识的聚合增强了蛋白质语言模型在各种应用程序中的性能.
Minh Hoang1, Mona Singh1,2
1Lewis-Sigler Institute of Integrative Genomics, Princeton University, Princeton, NJ 08540, United States.
Bioinformatics (Oxford, England)
|July 15, 2025
概括
蛋白质语言模型 (PLM) 从新的注意力聚合方法中获益,以改进蛋白质序列分析. 鱼袋聚合 (BoM-Pooling) 提高了效率,并捕获了重要的本地和远程蛋白质特征.
科学领域:
- 计算生物学是一种计算生物学.
- 生物信息学是一种生物信息学.
- 机器学习在生物学中的应用
背景情况:
- 蛋白质语言模型 (PLM) 使用变压器架构分析蛋白质序列.
- 目前的PLM产生了上下文化的氨基酸表示.
- 对于下游任务,每残留嵌入被聚合到固定尺寸向量中,但像Cls-Pooling和Avg-Pooling这样的常用方法错过了局部子结构和远程相互作用.
研究的目的:
- 引入注意力聚合,以捕捉蛋白质中的本地基结构和长距离相互作用.
- 为了制定一个有效的聚合策略,Bag-of-mer聚合 (BoM-Pooling),通过将窗口平均聚合与注意力聚合相结合.
- 为了提高蛋白质序列建模的有效性,使用生物启发的聚合技术.
主要方法:
- 建议聚合注意力以捕捉蛋白质特征.
- 引入了袋子聚合 (BoM-Pooling),一种分层聚合技术.
- 结合窗口平均值的聚合与注意力聚合用于计算可行性.
主要成果:
- 注意力聚合和BoM聚合的表现优于传统的聚合策略.
- 在预测蛋白质活动,检测远程同类物和预测信号蛋白相互作用方面表现出卓越的表现.
- 突出了生物启发的聚合在蛋白质序列建模中的优点.
结论:
- 注意力聚合有效地捕捉了蛋白质的基本特征.
- 博姆聚合为PLM提供了一个高效和有效的聚合策略.
- 生物启发的聚合技术代表了生物应用中的语言模型的重大进步.
更多相关视频
12:11Simultaneous Affinity Enrichment of Two Post-Translational Modifications for Quantification and Site Localization
Published on: February 27, 2020
7.0K
09:09Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody
Published on: September 27, 2024
536
相关概念视频
02:07Improving Translational Accuracy
11.9K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
11.9K
02:26Conservation of Protein Domains Over Different Proteins
11.4K
Protein domains are small structurally independent units that are part of a single amino acid chain. Although these domains are often structurally independent, they may rely on synergistic effects to perform their functions as part of a larger protein. Protein domains may be conserved within the same organism, as well as across different organisms.
A limited set of protein domains often duplicate and recombine during evolution. These domains can be organized in different combinations to...
A limited set of protein domains often duplicate and recombine during evolution. These domains can be organized in different combinations to...
11.4K
01:22Language Development
455
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
455
02:28Leaky Scanning
5.2K
During most eukaryotic translation processes, the small 40S ribosome subunit scans an mRNA from its 5' end until it encounters the first start AUG codon. The large 60S ribosomal subunit then joins the smaller one to initiate protein synthesis. The location of the translation initiation is largely determined by the nucleotides near the start codon as there may be multiple translation initiation sites present on the mRNA. Marilyn Kozak discovered that the sequence RCCAUGG (where R...
5.2K
02:26Conservation of Protein Domains
3.2K
3.2K