相关实验视频

Updated: Jun 23, 2025

Protein WISDOM: A Workbench for In silico De novo Design of BioMolecules

Protein WISDOM: A Workbench for In silico De novo Design of BioMolecules

Published on: July 25, 2013

通过参数有效的微调来民主化蛋白质语言模型

Samuel Sledzieski^1,2, Meghana Kshirsagar¹, Minkyung Baek³

¹AI for Good Research Lab, Microsoft Corporation, Redmond, WA 98052.

Proceedings of the National Academy of Sciences of the United States of America

|June 20, 2024

概括

像LoRA这样的参数高效微调 (PEFT) 方法使大蛋白语言模型 (PLM) 民主化,用于蛋白质组学研究. 这些技术提供了具有竞争力的性能,减少了计算和内存需求,使得先进的蛋白质分析可供更多的研究人员使用.

科学领域:

计算生物学计算生物学
蛋白质组学是指蛋白质组学.
人工智能的人工智能

背景情况:

大型蛋白质语言模型 (PLM) 通过从大量序列数据中学习,改变了蛋白质组学.
PLM的传统微调 (FT) 需要大量的计算资源,限制了许多研究小组的可访问性.
参数高效微调 (PEFT) 方法已经解决了自然语言处理中的类似挑战.

研究的目的:

引入和评估PEFT方法以适应蛋白质组学中的PLM.
评估PEFT在蛋白质-蛋白质相互作用 (PPI) 预测和同类分子对称性预测方面的有效性.
为民主化蛋白质组学中的PLM适应提供资源.

主要方法:

在PLM的PEFT中利用了LoRA (低级别调整) 方法.
训练有素的模型用于预测蛋白质与蛋白质相互作用 (PPI) 和同类聚合物四元结构对称性.
在蛋白质组学中对PEFT进行了超参数空间的全面评估.

主要成果:

与传统的FT相比,PEFT方法在两个任务中都表现出了竞争力.
PEFT 方法需要更少的参数和更少的内存足迹.
仅训练PPI预测的分类头是非常有效的,使用的参数要少得多.
PEFT方法的性能优于最先进的PPI预测技术,计算成本降低.
发现PEFT对超参数变化具有稳定性.

结论:

在蛋白质组学中,PEFT为PLM提供了一个计算效率高和有效的替代传统FT.
这些方法使有限的计算资源的研究人员能够获得先进的PLM适应.
蛋白质组学中的PEFT最佳实践可能与自然语言处理中的最佳实践有所不同,需要进一步调查.

关键词:

同类寡合体对称性对称性具有参数效率的微调.蛋白质语言模型的模型蛋白质蛋白质相互作用四分制结构的四分制结构.

更多相关视频

Tuning Degradation to Achieve Specific and Efficient Protein Depletion

Tuning Degradation to Achieve Specific and Efficient Protein Depletion

Published on: July 20, 2019

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

相关概念视频

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving

Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...

Improving Translational Accuracy

Improving Translational Accuracy

Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...

Leaky Scanning

Leaky Scanning

During most eukaryotic translation processes, the small 40S ribosome subunit scans an mRNA from its 5' end until it encounters the first start AUG codon. The large 60S ribosomal subunit then joins the smaller one to initiate protein synthesis. The location of the translation initiation is largely determined by the nucleotides near the start codon as there may be multiple translation initiation sites present on the mRNA. Marilyn Kozak discovered that the sequence RCCAUGG (where R...

Mechanistic Models: Compartment Models in Individual and Population Analysis

Mechanistic Models: Compartment Models in Individual and Population Analysis

Mechanistic models are utilized in individual analysis using single-source data, but imperfections arise due to data collection errors, preventing perfect prediction of observed data. The mathematical equation involves known values (Xi), observed concentrations (Ci), measurement errors (εi), model parameters (ϕj), and the related function (ƒi) for i number of values. Different least-squares metrics quantify differences between predicted and observed values. The ordinary least...