Protein sequence modelling with Bayesian flow networks

Area of Science:

• Computational biology
• Protein engineering
• Machine learning

Background:

• Generative machine learning has advanced protein sequence modeling.
• Existing models struggle with both unconditional and conditional generation.
• Bayesian Flow Networks (BFNs) offer a novel framework for generative modeling.

Purpose of the Study:

• To introduce Bayesian Flow Networks (BFNs) for protein sequence generation.
• To develop and evaluate ProtBFN, a large-scale BFN model for protein sequences.
• To assess the capability of BFNs in conditional generation for antibody design.

Main Methods:

• Developed ProtBFN, a 650M parameter model based on Bayesian Flow Networks.
• Trained ProtBFN on protein sequences from UniProtKB.
• Fine-tuned ProtBFN on antibody heavy chains to create AbBFN for conditional generation tasks.

Main Results:

• ProtBFN generates natural-like, diverse, structurally coherent, and novel protein sequences.
• ProtBFN significantly outperforms leading autoregressive and discrete diffusion models.
• The antibody-specific model, AbBFN, demonstrates competitive or superior zero-shot conditional generation compared to BERT-style models.

Conclusions:

• Bayesian Flow Networks are highly effective for protein sequence generation.
• ProtBFN represents a significant advancement in generative protein modeling.
• BFNs show promise for specialized applications like antibody engineering and therapeutic protein design.