ProtMamba: a homology-aware but alignment-free protein state space model

Area of Science:

• Computational biology
• Protein engineering
• Bioinformatics

Background:

• Protein language models advance understanding of sequence-to-function relationships and protein design.
• Multiple sequence alignment-based models capture evolutionary information but face construction challenges.

Purpose of the Study:

• Introduce ProtMamba, a novel homology-aware, alignment-free protein language model.
• Evaluate ProtMamba's performance in various protein design applications.

Main Methods:

• Developed ProtMamba using the Mamba architecture, enabling efficient long-context handling.
• Trained ProtMamba on homologous sequences with a combined autoregressive and masked language modeling objective.
• Utilized a fill-in-the-middle training strategy for adaptability in protein design.

Main Results:

• ProtMamba demonstrates computational efficiency and handles hundreds of protein sequences.
• Achieved superior performance in homolog-conditioned sequence generation compared to state-of-the-art models.
• Showcased utility in sequence generation, motif inpainting, fitness prediction, and modeling disordered regions.

Conclusions:

• ProtMamba offers a powerful and efficient alternative for protein language modeling.
• Highlights the significance of long-context conditioning in protein design applications.
• The model's competitive performance underscores the potential of alignment-free approaches.