H3BERTa: A CDR-H3-specific language model for antibody repertoire analysis

Area of Science:

• Immunology
• Bioinformatics
• Computational Biology

Background:

• Predicting antibody function is difficult due to data limitations and sequence diversity.
• The CDR-H3 region is highly variable and crucial for antigen specificity.
• Existing deep learning models struggle with full antibody variable regions.

Purpose of the Study:

• To assess the biological information encoded solely within the CDR-H3 sequence.
• To develop a focused language model (H3BERTa) for CDR-H3 analysis.
• To explore the utility of CDR-H3 embeddings for antibody discovery and repertoire analysis.

Main Methods:

• Developed H3BERTa, a language model trained exclusively on CDR-H3 sequences.
• Utilized H3BERTa embeddings to capture immunological features.
• Applied pseudo-perplexity profiles for repertoire analysis.
• Trained classifiers for broadly neutralizing antibodies using limited data.

Main Results:

• H3BERTa embeddings successfully recapitulated J-gene usage, B cell maturation state, and antigen binding.
• Repertoire analysis distinguished healthy from HIV-1-derived sequences, identifying immune response signatures.
• Classifiers achieved high performance for broadly neutralizing antibodies with limited labeled data.

Conclusions:

• The CDR-H3 region alone contains a rich immunological signal.
• H3BERTa effectively captures this signal, providing a focused tool for analysis.
• This approach enhances antibody discovery, engineering, and repertoire analysis.