一个课程学习方法来训练抗体语言模型
Sarah M Burbach1, Bryan Briney1,2,3,4,5
1Department of Immunology and Microbiology, The Scripps Research Institute, La Jolla, California, United States of America.
PLoS computational biology
|September 11, 2025
概括
我们为抗体语言模型 (AbLMs) 引入课程学习,以有效地训练混合未配对和配对序列数据. 这种方法CurrAb在关键抗体预测任务中优于现有模型.
科学领域:
- 计算生物学 计算生物学
- 生物信息学是一种生物信息学.
- 机器学习 机器学习
背景情况:
- 抗体语言模型 (AbLMs) 越来越多地使用未配对和配对抗体序列进行预训练.
- 一个关键的挑战是优化培训策略混合数据集由于大小不平衡.
研究的目的:
- 系统地评估数据处理和培训策略,以混合序列数据进行培训的ABLM.
- 为ABLM预培训引入和优化课程学习方法.
主要方法:
- 为AbLMs开发了一个课程学习方法,逐渐从不配对到配对的序列过渡.
- 比较课程学习与不断混合和微调数据采样策略.
- 对下游残留物预测和分类任务的评估模型性能.
主要成果:
- 课程学习和不断混合方法在大规模模型中表现优于微调.
- 这些方法减轻了灾难性的遗忘和缓慢过度装配.
- 一个650M参数的课程模型,CurrAb,实现了卓越的性能.
结论:
- 课程学习是对不平衡的混合数据集进行ABLM预培训的有效策略.
- 在抗体序列分析任务中,CurrAb展示了最先进的性能.
- 优化训练策略对于利用大规模抗体序列数据至关重要.
相关概念视频
Antibody Structure and Classes
8.4K
Antibodies, also known as immunoglobulins, are produced by B cells in response to foreign substances, such as bacteria and viruses. These proteins are critical for recognizing and neutralizing these substances, protecting the body from potential harm.
The basic structure of an antibody consists of four protein chains: two identical heavy chains and two identical light chains. These chains are held together by disulfide bonds and other non-covalent interactions, forming a Y-shaped structure.
The basic structure of an antibody consists of four protein chains: two identical heavy chains and two identical light chains. These chains are held together by disulfide bonds and other non-covalent interactions, forming a Y-shaped structure.
8.4K
Antibody Structure
65.3K
Overview
Antibodies, also known as immunoglobulins (Ig), are essential players of the adaptive immune system. These antigen-binding proteins are produced by B cells and make up 20 percent of the total blood plasma by weight. In mammals, antibodies fall into five different classes, which each elicits a different biological response upon antigen binding.
The Y-Shaped Structure of Antibodies Consists of Four Polypeptide Chains
Antibodies consist of four polypeptide chains: two identical heavy...
Antibodies, also known as immunoglobulins (Ig), are essential players of the adaptive immune system. These antigen-binding proteins are produced by B cells and make up 20 percent of the total blood plasma by weight. In mammals, antibodies fall into five different classes, which each elicits a different biological response upon antigen binding.
The Y-Shaped Structure of Antibodies Consists of Four Polypeptide Chains
Antibodies consist of four polypeptide chains: two identical heavy...
65.3K
Antibody Actions
2.4K
Antibodies, or immunoglobulins, are critical players in the immune system's arsenal against invading pathogens. Produced by B cells and plasma cells, their primary role is to detect and bind to specific antigens, molecules found on the surface of pathogens like bacteria or viruses. Beyond antigen recognition, antibodies perform several vital functions that contribute to immune defense.
Neutralization
Antibodies can bind to pathogens, preventing them from infecting host cells. This process...
Neutralization
Antibodies can bind to pathogens, preventing them from infecting host cells. This process...
2.4K
Immunological Memory
15.1K
Immunological memory, a pivotal pillar of the adaptive immune system, is responsible for the body's ability to remember and respond more swiftly and effectively to previously encountered pathogens. This remarkable feature is what makes vaccines so effective in preventing diseases.
What is Immunological Memory?
Immunological memory is an integral function of the immune system that allows it to recognize and react more rapidly and effectively to pathogens previously encountered. This feature...
What is Immunological Memory?
Immunological memory is an integral function of the immune system that allows it to recognize and react more rapidly and effectively to pathogens previously encountered. This feature...
15.1K
Purposive Learning
451
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
451
Language Development
855
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
855


