PETA:评估蛋白质转移学习与子词令牌化对下游应用程序的影响
Yang Tan1,2,3,4, Mingchen Li1,2,3,4, Ziyi Zhou2
1School of Information Science and Engineering, East China University of Science and Technology, Shanghai, 200237, China.
Journal of cheminformatics
|August 2, 2024
概括
本研究为蛋白质语言模型 (PLM) 引入了先进的蛋白质序列标记化. 通过使用字节对编码和单程,新的词汇提高了下游任务的PLM性能,并建立了评估的基准.
科学领域:
- 计算生物学 计算生物学
- 生物信息学是一种生物信息学.
- 机器学习 机器学习
背景情况:
- 蛋白质语言模型 (PLM) 对于蛋白质表示学习至关重要.
- 当前的PLM通常将氨基酸视为孤立的标记,忽视生物背景.
- 这种方法忽略了经常同时发生的氨基酸组合,限制了模型的有效性.
研究的目的:
- 开发先进的蛋白质序列令牌化方法.
- 通过结合生物相关的残留物组合来提高PLM的性能.
- 引入一个全面的基准来评估PLM.
主要方法:
- 利用字节对编码和单图算法进行先进的残留词汇构建.
- 预先训练有素的PLM使用这些新词汇.
- 开发了PETA,这是系统PLM评估的基准.
主要成果:
- 与使用简单词汇相比,使用先进词汇预先训练的PLM在下游任务中表现优异.
- 发现含有50个和200个元素的词汇表产生了最佳的性能.
- 该PETA基准指标促进了对PLM性能进行系统评估.
结论:
- 先进的令牌化策略显著改善了蛋白质表示学习.
- 词汇大小的选择对于最佳的PLM性能至关重要.
- 该PETA基准为评估PLM提供了一个标准化的框架.
相关概念视频
Conservation of Protein Domains Over Different Proteins
10.8K
Protein domains are small structurally independent units that are part of a single amino acid chain. Although these domains are often structurally independent, they may rely on synergistic effects to perform their functions as part of a larger protein. Protein domains may be conserved within the same organism, as well as across different organisms.
A limited set of protein domains often duplicate and recombine during evolution. These domains can be organized in different combinations to...
A limited set of protein domains often duplicate and recombine during evolution. These domains can be organized in different combinations to...
10.8K
Transfer RNA Synthesis
11.9K
One of the unique features of tRNA is the presence of modified bases. In some tRNAs, modified bases account for nearly 20% of the total bases in the molecule. Altogether, these unusual bases protect the tRNA from enzymatic degradation by RNases.
Each of these chemical modifications is carried by a specific enzyme, post-transcription. All of these enzymes have unique base and site-specificity. Methylation, the most common chemical modification, is carried by at least nine different enzymes, with...
Each of these chemical modifications is carried by a specific enzyme, post-transcription. All of these enzymes have unique base and site-specificity. Methylation, the most common chemical modification, is carried by at least nine different enzymes, with...
11.9K
Improving Translational Accuracy
2.5K
2.5K
Conservation of Protein Domains
3.1K
3.1K
Cotranslational Protein Translocation
7.3K
Translocation of proteins across membranes is an ancient process that occurs even in bacteria and archaebacteria. In fact, the components of the translocation machinery are still conserved between prokaryotes and eukaryotes.
Sec61 channel partners for cotranslational translocation
During cotranslational translocation, the Sec61 channel partners with the signal recognition particle (SRP), the signal recognition particle receptor (SR), and the ribosomes to transport the nascent polypeptide chain...
Sec61 channel partners for cotranslational translocation
During cotranslational translocation, the Sec61 channel partners with the signal recognition particle (SRP), the signal recognition particle receptor (SR), and the ribosomes to transport the nascent polypeptide chain...
7.3K
Protein Networks
2.3K
2.3K


