BERT2DAb: a pre-trained model for antibody representation based on amino acid sequences and 2D-structure

Area of Science:

• Computational Biology
• Immunology
• Bioinformatics

Background:

• Large antibody sequence datasets enable pre-trained language models for antibody screening and optimization.
• Fewer pre-trained models exist for antibody sequences compared to general proteins.
• Existing models lack explicit secondary structure feature integration.

Purpose of the Study:

• Introduce BERT2DAb, a novel pre-trained language model for antibody sequences.
• Incorporate secondary structure information using self-attention mechanisms.
• Improve antibody sequence representation and downstream task performance.

Main Methods:

• Developed BERT2DAb, a pre-trained model utilizing self-attention for secondary structure integration.
• Evaluated model performance on antigen-antibody binding classification tasks.
• Assessed model on antigen-antibody complex mutation binding free energy prediction.
• Proposed a method to analyze attention weights and tertiary structure contact states for interpretability.

Main Results:

• Achieved state-of-the-art performance on three downstream tasks.
• Demonstrated high precision (85.15%/94.86%) and recall (87.41%/86.15%) in binding classification.
• Obtained a high Pearson correlation coefficient (0.77) for binding free energy prediction.
• Enhanced model interpretability through novel attention weight analysis.

Conclusions:

• BERT2DAb effectively leverages secondary structure information for antibody sequence representation.
• The model shows significant potential for advancing antibody screening and design.
• Improved interpretability offers deeper insights into antibody-protein interactions.