精细调整的蛋白质语言模型可以提高跨多种任务的预测
Robert Schmirler1,2,3, Michael Heinzinger4, Burkhard Rost4,5,6
1TUM (Technical University of Munich), School of Computation, Information and Technology (CIT), Faculty of Informatics, Chair of Bioinformatics & Computational Biology - i12, Garching/Munich, Germany. robert.schmirler@tum.de.
Nature communications
|August 28, 2024
概括
微调蛋白语言模型 (PLM) 显著提高了各种任务的预测准确性. 参数高效的微调提供了一个节约资源的替代方案,加速培训并实现可比结果.
科学领域:
- 计算生物学 计算生物学
- 生物信息学是一种生物信息学.
- 机器学习 机器学习
背景情况:
- 蛋白质语言模型 (PLMs) 产生嵌入,实现蛋白质预测的最先进性能.
- 与自然语言处理不同,PLM嵌入通常没有为下游蛋白质预测任务进行微调.
研究的目的:
- 研究微调对最先进的蛋白质语言模型 (ESM2,ProtT5,Ankh) 的影响.
- 在八个不同的蛋白质预测任务中比较完整模型微调与参数有效微调 (PEFT).
主要方法:
- 评估了三种主要的蛋白质语言模型:ESM2,ProtT5和Ankh.
- 对每个模型应用特定任务的监督微调.
- 将全参数微调与参数高效微调策略进行比较.
- 评估了八个不同的蛋白质预测基准的性能.
主要成果:
- 特定任务的微调始终改善了下游预测性能.
- 参数高效的微调实现了可比的改进,显著减少了计算资源.
- 与完整模型微调相比,PEFT加速训练的速度高达4.5倍.
结论:
- 微调蛋白质语言模型是强烈推用于提高预测准确性,特别是对于小型数据集.
- 参数效率微调为适应PLM提供了一个计算效率高和有效的方法.
- 该研究提供了可访问的笔记本,用于微调各种PLM用于每蛋白和每残留预测任务.
相关概念视频
02:07Improving Translational Accuracy
9.4K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
9.4K
02:28Leaky Scanning
5.1K
During most eukaryotic translation processes, the small 40S ribosome subunit scans an mRNA from its 5' end until it encounters the first start AUG codon. The large 60S ribosomal subunit then joins the smaller one to initiate protein synthesis. The location of the translation initiation is largely determined by the nucleotides near the start codon as there may be multiple translation initiation sites present on the mRNA. Marilyn Kozak discovered that the sequence RCCAUGG (where R...
5.1K
01:27Language and Cognition
338
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
338
01:29Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
45
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
45
01:23Regulation of Expression at Multiple Steps
875
The gene expression in cells is regulated at different stages: (i) transcription, (ii) RNA processing, (iii) RNA localization, and (iv) translation. Transcriptional regulation is mediated by regulatory proteins such as transcription factors, activators, or repressors—these control gene expression by initiating or inhibiting the transcription of genes. Once a precursor or pre-mRNA is produced, it undergoes post-transcriptional modification, including 5' capping, splicing, and the...
875