A tri-modal contrastive learning framework for protein representation learning

Area of Science:

• Biochemistry
• Computational Biology
• Structural Biology

Background:

• Protein foundation models, particularly language models, excel at learning representations from amino acid sequences using self-supervised learning on large datasets.
• These sequence-based representations are effective for predicting protein functions and properties.
• Current models often neglect crucial data like 3D structures and scientific literature, limiting their scope.

Purpose of the Study:

• To develop a multimodal pretraining framework that integrates protein sequences, 3D structures, and literature text.
• To address the limitations of existing models in modality coverage and training strategies.
• To capture richer and more holistic protein representations by leveraging complementary data sources.

Main Methods:

• Proposed a multimodal pretraining framework integrating three modalities: protein sequences, 3D structures, and literature text.
• Utilized protein sequences as the anchor modality.
• Employed contrastive learning to align structural and textual modalities with the sequence modality.

Main Results:

• The developed framework, ProteinAligner, captures more comprehensive protein representations.
• ProteinAligner demonstrated superior performance across a diverse range of downstream tasks compared to existing state-of-the-art foundation models.
• The model showed significant improvements in predicting protein functions and properties.

Conclusions:

• Integrating multiple modalities (sequences, structures, text) enhances protein representation learning.
• The proposed multimodal framework effectively captures holistic protein information.
• ProteinAligner represents a significant advancement in foundation models for protein science, improving predictive accuracy for biological functions and properties.