Improving antibody language models with native pairing.

Area of Science:

• Computational biology
• Immunoinformatics
• Machine learning in immunology

Background:

• Current antibody language models often utilize unpaired sequence data, limiting their ability to capture crucial inter-chain relationships.
• A large dataset of natively paired human antibody sequences provides a novel resource for advancing antibody modeling.

Purpose of the Study:

• To evaluate the impact of training antibody language models with natively paired sequences versus unpaired or randomly paired sequences.
• To assess if native pairing enables models to learn cross-chain immunological features and improve performance on downstream tasks.

Main Methods:

• Trained three baseline antibody language models (BALM) using natively paired, randomly-paired, and unpaired sequences.
• Fine-tuned the Evolutionary Scale Modeling (ESM)-2 model with natively paired antibody sequences.
• Evaluated model performance on various metrics, including classification of antibodies by pathogen specificity.

Main Results:

• Models trained with natively paired sequences learned immunologically relevant features spanning both light and heavy chains.
• Native pairing significantly improved model performance compared to random or unpaired training.
• Enhanced ability to classify antibodies based on pathogen specificity was observed.

Conclusions:

• Training antibody language models with natively paired sequences is superior to using unpaired or randomly paired data.
• Native pairing facilitates the learning of critical inter-chain antibody features, leading to improved predictive power.
• This methodology advances the development of more accurate and effective antibody prediction models.