一个课程学习方法来训练抗体语言模型
Sarah M Burbach1, Bryan Briney1,2,3,4,5
1Department of Immunology and Microbiology, The Scripps Research Institute, La Jolla, CA 92037 USA.
bioRxiv : the preprint server for biology
|March 10, 2025
概括
我们为抗体语言模型 (AbLMs) 引入课程学习,以有效地训练混合未配对和配对序列数据. 我们的优化方法CurrAb在分类任务中表现出与现有模型相比的卓越性能.
科学领域:
- * 计算生物学 计算生物学
- * 生物信息学是一门学科.
- * 免疫学中的机器学习
背景情况:
- *抗体语言模型 (AbLMs) 越来越多地使用未配对和配对抗体序列进行预训练.
- *结合这些数据集提供了潜在的好处,但由于数据不平衡和缺乏对数据处理和培训策略的系统评估,因此面临挑战.
- *现有的方法很难充分利用未配对数据的规模,并与配对数据的特异性相结合.
研究的目的:
- * 开发和评估一种系统的方法,用于预训练AbLMs,使用未配对和配对抗体序列的混合物.
- * 解决与大型未配对数据集与较小,高质量的配对数据集相结合所固有的数据不平衡挑战.
- * 优化训练策略,最大限度地提高两种数据类型的实用性,以提高AbLM性能.
主要方法:
- *为ABLMs引入课程学习方法,使分阶段的培训过程从不配对到配对的序列.
- *优化数据处理和培训方法,为混合数据ABLM预培训量身定制.
- * 开发和评估一个名为CurrAb.Ab的650M参数模型.
主要成果:
- *开发的课程学习方法在ABLM预训练期间有效地整合了未配对和配对的抗体序列.
- *使用这种方法的CurrAb模型在下游分类任务上明显优于现有的混合ABLM.
- *优化的培训策略成功地减轻了数据不平衡带来的挑战.
结论:
- *课程学习提供了一种可行和有效的策略,用于在混合序列数据集上预培训ABLM.
- *CurrAb模型为基于抗体序列的分类任务的性能建立了新的基准.
- * 这项工作为未来开发AbLMs提供了基础框架,利用各种抗体序列数据.
相关概念视频
Antibody Structure and Classes
724
Antibodies, also known as immunoglobulins, are produced by B cells in response to foreign substances, such as bacteria and viruses. These proteins are critical for recognizing and neutralizing these substances, protecting the body from potential harm.
The basic structure of an antibody consists of four protein chains: two identical heavy chains and two identical light chains. These chains are held together by disulfide bonds and other non-covalent interactions, forming a Y-shaped structure.
The basic structure of an antibody consists of four protein chains: two identical heavy chains and two identical light chains. These chains are held together by disulfide bonds and other non-covalent interactions, forming a Y-shaped structure.
724
Antibody Structure
58.7K
Overview
Antibodies, also known as immunoglobulins (Ig), are essential players of the adaptive immune system. These antigen-binding proteins are produced by B cells and make up 20 percent of the total blood plasma by weight. In mammals, antibodies fall into five different classes, which each elicits a different biological response upon antigen binding.
The Y-Shaped Structure of Antibodies Consists of Four Polypeptide Chains
Antibodies consist of four polypeptide chains: two identical heavy...
Antibodies, also known as immunoglobulins (Ig), are essential players of the adaptive immune system. These antigen-binding proteins are produced by B cells and make up 20 percent of the total blood plasma by weight. In mammals, antibodies fall into five different classes, which each elicits a different biological response upon antigen binding.
The Y-Shaped Structure of Antibodies Consists of Four Polypeptide Chains
Antibodies consist of four polypeptide chains: two identical heavy...
58.7K
Hybridoma Technology
13.9K
Hybridoma technology is used for the large-scale production of monoclonal antibodies. Monoclonal antibodies bind to only a single antigenic determinant or epitope. Such antibodies are used in research, diagnostics, and disease therapy. The hybridoma technology established in 1975 by Georges Köhler and Cesar Milstein was awarded the Nobel Prize in Medicine in 1984 for revolutionizing research and therapy.
Hybridoma Selection
Commonly used fusion techniques — electroporation,...
Hybridoma Selection
Commonly used fusion techniques — electroporation,...
13.9K
Active versus Passive Immunity
523
Immunity, along with the ability to limit pathogen growth to prevent significant body tissue damage, can be gained either by (1) actively developing an immune response within the individual after exposure to a pathogen or after getting vaccinated or (2) passively transferring immune components from an immune individual to one who is nonimmune. Both these forms of immunity can be found naturally and in medical practices.
Active Immunity
Active immunity refers to the resistance one develops...
Active Immunity
Active immunity refers to the resistance one develops...
523
Immunological Memory
514
Immunological memory, a pivotal pillar of the adaptive immune system, is responsible for the body's ability to remember and respond more swiftly and effectively to previously encountered pathogens. This remarkable feature is what makes vaccines so effective in preventing diseases.
What is Immunological Memory?
Immunological memory is an integral function of the immune system that allows it to recognize and react more rapidly and effectively to pathogens previously encountered. This feature...
What is Immunological Memory?
Immunological memory is an integral function of the immune system that allows it to recognize and react more rapidly and effectively to pathogens previously encountered. This feature...
514


