面具语言建模下游的预训练动态:T细胞受体结合预测
1Division of Computer Science & Engineering, Louisiana State University, Baton Rouge, LA 70803, United States.
Bioinformatics advances
|March 17, 2025
概括
蒙面语言建模改善了:T细胞受体结合预测. 在预训练损失趋同之前,性能增长达到峰值,使损失成为最佳模型检查点的不可靠指标.
科学领域:
- 免疫信息学是指免疫信息学.
- 计算生物学是一种计算生物学.
- 机器学习 机器学习
背景情况:
- 由于庞大的组合和有限的结合数据,预测抗原和T细胞受体 (TCR) 的结合具有挑战性.
- 蒙面语言建模 (MLM) 预训练通过使用未标记的数据来增强:TCR结合预测模型.
研究的目的:
- 为了研究在基于变压器的的MLM预训期间实现较低损失指标的好处:TCR结合预测模型.
- 评估训练前损失的收是否准确地反映了下游的最佳性能.
主要方法:
- 变压器模型架构使用面具语言建模进行了预训练.
- 下游绩效指标是在连续的预训练间隔记录的.
- 该研究分析了训练前损失和预测性表现之间的关系.
主要成果:
- 业绩效益从MLM预训练高原显著在预训练损失收之前.
- 预培训损失是识别下游任务最佳模型检查点的无效指标.
- 预培训损失可以表明当进一步的预培训产生收益时,收益率会下降.
结论:
- 根据损失的收优化预训时间对于:TCR结合预测不是理想的.
- 超过收益下降点的进一步预训练不会损害表现,但不会提供额外的好处.
- 精心选择预训练间隔对于最大化模型性能至关重要.
更多相关视频
09:32Immunopeptidomics: Isolation of Mouse and Human MHC Class I- and II-Associated Peptides for Mass Spectrometry Analysis
Published on: October 15, 2021
11.5K
06:50Author Spotlight: A Computational Approach to Decipher Amino Acid Preferences in Multispecific Protein-Protein Interactions
Published on: January 26, 2024
1.0K
相关概念视频
02:07Improving Translational Accuracy
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
02:07Improving Translational Accuracy
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...