面具语言建模用于资源受限的生物自然语言处理
概括
我们使用面具语言建模开发了一种新的蛋白质序列嵌入方法. 这种方法显著提高了对二次结构和内在无序区域预测任务的性能,即使资源有限.
科学领域:
- 生物信息学是一种生物信息学.
- 计算生物学 计算生物学
- 自然语言处理自然语言处理.
背景情况:
- 最近的自然语言处理 (NLP) 进步,特别是深度上下文嵌入器,已经超越了传统的词向量模型.
- 由20个氨基酸组成的蛋白质序列可以被视为一种自然语言,对于理解生物功能至关重要.
- 有效的矢量表示和模型架构对于生物任务至关重要.
研究的目的:
- 使用未标记数据开发一种高效的蛋白质序列嵌入方法.
- 为了评估一种新型的卷积和门式循环网络 (CGRN) 嵌入器的性能,该嵌入器经过了面具语言建模 (MLM) 的训练.
- 分析不同模型架构对下游生物任务的影响.
主要方法:
- 使用未标记的蛋白质序列进行训练.
- 采用了卷积和门式循环网络 (CGRN) 架构.
- 应用了面具语言建模 (MLM) 技术用于嵌入器培训.
主要成果:
- 在二次结构预测 (SSP) 上获得了73.1%的F1分数 (Q8).
- 在内在无序区域预测 (IDRP) 中获得了84%的F1分数.
- 在资源有限的环境下,经过显著的表现提升.
结论:
- 经过MLM培训的CGRN嵌入器在SSP和IDRP等下游任务上表现出色.
- 模型架构的选择显著影响基于特定的生物任务的性能.
- 这种方法为蛋白质序列分析提供了有效的解决方案,特别是在资源有限的环境中.
更多相关视频
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
231
10:41Leveraging CyVerse Resources for De Novo Comparative Transcriptomics of Underserved Non-model Organisms
Published on: May 9, 2017
9.3K
相关概念视频
Leaky Scanning
5.1K
During most eukaryotic translation processes, the small 40S ribosome subunit scans an mRNA from its 5' end until it encounters the first start AUG codon. The large 60S ribosomal subunit then joins the smaller one to initiate protein synthesis. The location of the translation initiation is largely determined by the nucleotides near the start codon as there may be multiple translation initiation sites present on the mRNA. Marilyn Kozak discovered that the sequence RCCAUGG (where R...
5.1K
Mechanistic Models: Compartment Models in Individual and Population Analysis
43
Mechanistic models are utilized in individual analysis using single-source data, but imperfections arise due to data collection errors, preventing perfect prediction of observed data. The mathematical equation involves known values (Xi), observed concentrations (Ci), measurement errors (εi), model parameters (ϕj), and the related function (ƒi) for i number of values. Different least-squares metrics quantify differences between predicted and observed values. The ordinary least...
43
Nonsense-mediated mRNA Decay
10.6K
The Upf proteins that carry out nonsense-mediated decay (NMD) are found in all eukaryotic organisms, including humans. Each protein has an individual role, but they need to work in collaboration. Upf1 is an ATP-dependent RNA helicase that unwinds the RNA helix. Because Upf1 can unwind any RNA, Upf2 and Upf3 are required to help Upf1 discriminate between nonsense and normal mRNAs.
Usually, Upf3 binds to an Exon Junction Complex (EJC) at mRNA splice sites. If a ribosome fully translates the mRNA,...
Usually, Upf3 binds to an Exon Junction Complex (EJC) at mRNA splice sites. If a ribosome fully translates the mRNA,...
10.6K
