相关实验视频

Updated: Jan 17, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

对配对抗体语言模型的数据最佳缩放

Mahdi Shafiei Neyestanak^1,2, Sarah M Burbach¹, Karenna Ng¹

¹Department of Immunology and Microbiology, The Scripps Research Institute, La Jolla, CA 92037 USA.

bioRxiv : the preprint server for biology

|September 15, 2025

概括

抗体语言模型 (AbLMs) 是数据有限的,与计算受约束的自然语言模型不同. 最佳的AbLM性能需要平衡模型大小和训练数据量,以改善下游任务结果.

科学领域:

计算生物学是一种计算生物学.
人工智能的人工智能是人工智能.
生物信息学是一种生物信息学.

背景情况:

大型语言模型 (LLM) 的缩放规律通常假定计算约束.
在配对序列上训练的抗体语言模型 (AbLMs) 面临数据限制.
这就需要对ABLM进行不同的扩展考虑.

研究的目的:

调查模型大小和数据规模对ABLM性能的影响.
为了推导出一个特定于AbLM的缩放定律.
确定对数据优化ABLMs的数据要求.

主要方法:

在5个模型大小和3个训练数据大小中训练了15个ABLM.
在下游分类任务上评估了ABLM的绩效.
导出了一个AbLM特定的缩放定律.

主要成果:

AbLM的性能与计算受限的LLM不同,主要是数据受限.
显著的性能增长需要足够大的模型尺寸.
相当于ESM-2 (650M参数) 的AbLM需要大约550万对抗体序列.

结论:

在数据有限的领域,如抗体建模,性能改进取决于模型规模和数据量.
目前对自然语言模型的缩放规则并不直接适用于AbLMs.
未来的AbLM开发应该专注于优化模型和数据尺度之间的相互作用.

关键词:

抗体语言模型的模型.深度学习是一种深度学习.免疫学免疫学免疫学缩放规则的法律,缩放规则的法律.变压器变压器

更多相关视频

Author Spotlight: Advancing Biotherapeutic Mass Calculation by Introducing mAbScale, a Python-Based Desktop Application

Author Spotlight: Advancing Biotherapeutic Mass Calculation by Introducing mAbScale, a Python-Based Desktop Application

Published on: June 16, 2023

Scalable High Throughput Selection From Phage-displayed Synthetic Antibody Libraries

Scalable High Throughput Selection From Phage-displayed Synthetic Antibody Libraries

Published on: January 17, 2015

相关概念视频

Improving Translational Accuracy

Improving Translational Accuracy

Improving Translational Accuracy

Improving Translational Accuracy

Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...

Model Approaches for Pharmacokinetic Data: Distributed Parameter Models

Model Approaches for Pharmacokinetic Data: Distributed Parameter Models

Pharmacokinetic models are mathematical constructs that represent and predict the time course of drug concentrations in the body, providing meaningful pharmacokinetic parameters. These models are categorized into compartment, physiological, and distributed parameter models.
The distributed parameter models are specifically designed to account for variations and differences in some drug classes. This model is particularly useful for assessing regional concentrations of anticancer or...