伯特:一种基于变压器的语言模型,用于预测性质.
Chakradhar Guntuboina1, Adrita Das2, Parisa Mollaei3
1Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213, United States.
The journal of physical chemistry letters
|November 13, 2023
概括
新的蛋白质语言模型PeptideBERT使用变压器技术准确地预测质的特性,如血液溶解和非污染. 这可以在不需要结构数据的情况下推进序列到属性预测.
科学领域:
- 生物技术是生物技术.
- 计算生物学 计算生物学
- 生物信息学是一种生物信息学.
背景情况:
- 大型语言模型 (LLM) 通过使序列表示成为文本来改变蛋白质建模.
- 这便于对的序列-属性预测,绕过了对明确结构数据的需求.
- 最近的LLM进步激发了分析的新应用.
研究的目的:
- 介绍PeptideBERT,这是一个专门的蛋白质语言模型,用于预测关键的特性.
- 评估PeptideBERT在预测血液溶解,溶解性和非污染特征方面的有效性.
- 为了利用基于变压器的模型进行增强的性质预测.
主要方法:
- 使用了ProtBERT预训练的变压器模型,具有12个注意力头和12个隐藏层.
- 在三个下游任务上微调了ProtBERT模型:血液溶解,可溶性和非污染预测.
- 主要使用较短的序列和数据集,重点关注不溶性阴性.
主要成果:
- 在预测红细胞相互作用的关键因素 - - 溶解方面取得了最先进的 (SOTA) 性能.
- 在预测非污染性质方面表现出了显著的表现.
- 展示了模型的有效性,尽管使用较短的序列和特定的负样本分布.
结论:
- BERT代表了使用语言模型预测基本性质的重大进步.
- 该模型在血液溶解和非污染预测中的SOTA性能突出显示了其潜在的实用性.
- 这种方法证明了基于变压器的模型在酸中的序列到属性预测中的力量.
相关概念视频
Peptide Identification Using Tandem Mass Spectrometry
6.5K
Tandem mass spectrometry, also known as MS/MS or MS2, is an analytical technique that employs two mass analyzers. Essentially it is a series of mass spectrometers that helps isolate a particular biomolecule and then helps study its chemical properties.
This technique helps gather information regarding the protein from which the peptide was obtained and to study the peptides’ amino acid sequence. Identifying peptides from a complex mixture is an important component of the growing field of...
This technique helps gather information regarding the protein from which the peptide was obtained and to study the peptides’ amino acid sequence. Identifying peptides from a complex mixture is an important component of the growing field of...
6.5K
Peptide Bonds
74.6K
A peptide bond covalently attaches amino acids through a dehydration reaction. One amino acid's carboxyl group and another amino acid's amino group combine, releasing a water molecule. The resulting bond is the peptide bond. The products that such linkages form are peptides. As more amino acids join this growing chain, the resulting chain is a polypeptide. Each polypeptide has a free amino group at one end. This end has the N-terminal, or the amino-terminal, and the other end has a free...
74.6K
Transformers
1.1K
A device that transforms voltages from one value to another using induction is called a transformer. A transformer consists of two separate coils, or windings, wrapped around the same soft iron core. However, they are electrically insulated from each other.
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
1.1K
Types Of Transformers
987
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
987


