Enhancing Structure-Aware Protein Language Models with Efficient Fine-Tuning for Various Protein Prediction Tasks

Area of Science:

• Computational biology
• Structural bioinformatics
• Machine learning for protein science

Background:

• Protein Language Models (PLMs) are vital for understanding and engineering proteins.
• Current PLMs primarily rely on protein sequences, neglecting crucial 3D structural information.
• This limitation hinders accurate prediction of protein interactions and functions.

Purpose of the Study:

• To develop a 3D structure-aware protein language model (S-PLM).
• To integrate protein sequence and 3D structure data into a unified latent space.
• To enhance protein prediction and design capabilities by incorporating structural insights.

Main Methods:

• Employed multi-view contrastive learning to align protein sequences with 3D structures.
• Utilized a geometric vector perceptron (GVP) model to process 3D coordinates and generate structural embeddings.
• Applied efficient fine-tuning methods for optimal performance on protein-related tasks.

Main Results:

• S-PLM significantly outperforms sequence-only PLMs in protein clustering and classification tasks.
• Achieved performance comparable to state-of-the-art methods that utilize both sequence and structure inputs.
• Demonstrated the efficacy of 3D structure integration for improved protein modeling.

Conclusions:

• S-PLM represents a significant advancement in protein language modeling by incorporating 3D structural information.
• Structure-aware models offer a powerful approach for enhancing protein understanding and engineering.
• The developed model provides a valuable tool for various protein-related applications.