Deciphering the language of antibodies using self-supervised learning.

Area of Science:

• Immunoinformatics
• Computational Biology
• Artificial Intelligence in Medicine

Background:

• B cell receptor (BCR) repertoires contain critical information on immune history and disease potential.
• Analyzing BCR sequences is vital for understanding disease and developing diagnostics and therapeutics.
• Predicting BCR properties solely from amino acid sequences presents a significant analytical challenge.

Purpose of the Study:

• To introduce Antibody-specific Bidirectional Encoder Representation from Transformers (AntiBERTa), a novel language model for BCR sequences.
• To demonstrate AntiBERTa's capability to generate contextualized representations (embeddings) of BCR sequences.
• To evaluate the utility of AntiBERTa embeddings for downstream applications, including predicting antibody features.

Main Methods:

• Development of AntiBERTa, a deep, antibody-specific language model based on the Transformer architecture.
• Pre-training AntiBERTa on extensive BCR sequence datasets to learn contextual representations.
• Fine-tuning AntiBERTa for the specific task of predicting paratope positions from antibody sequences.

Main Results:

• AntiBERTa embeddings effectively capture biologically relevant information within BCR sequences.
• The model's representations are generalizable across various immunoinformatics applications.
• Fine-tuned AntiBERTa significantly outperforms existing tools in predicting antibody paratope positions.

Conclusions:

• AntiBERTa represents the deepest protein-family-specific language model to date, offering rich BCR representations.
• AntiBERTa embeddings are highly versatile and can be readily applied to multiple downstream tasks.
• This model enhances our understanding of antibody sequence language and its implications for immunology and medicine.