PTM-Mamba: a PTM-aware protein language model with bidirectional gated Mamba blocks

Area of Science:

• Proteomics
• Bioinformatics
• Computational Biology

Background:

• Current protein language models (LMs) excel at encoding protein properties but lack representation of post-translational modifications (PTMs).
• PTMs are critical for proteomic diversity, influencing protein structure, function, and interactions.
• Existing models do not fully capture the complexity introduced by PTMs.

Purpose of the Study:

• To develop a PTM-aware protein language model that integrates PTM information.
• To enable accurate modeling of both wild-type and PTM-modified protein sequences.
• To establish a foundational tool for PTM-aware protein modeling and design.

Main Methods:

• Developed PTM-Mamba, a novel protein LM architecture.
• Integrated PTM tokens using bidirectional Mamba blocks.
• Fused PTM tokens with ESM-2 protein LM embeddings via a novel gating mechanism.

Main Results:

• PTM-Mamba uniquely models wild-type and PTM sequences.
• Enabled downstream tasks including disease association and druggability prediction.
• Facilitated prediction of PTM effects on protein-protein interactions and zero-shot PTM discovery.

Conclusions:

• PTM-Mamba represents a significant advancement in protein language modeling.
• The model provides a powerful tool for understanding PTMs' role in protein function.
• Established PTM-Mamba as a foundational resource for PTM-aware protein research and design.