Bilingual language model for protein sequence and structure

Area of Science:

• Computational biology
• Structural bioinformatics
• Machine learning in genomics

Background:

• Protein language models (pLMs) have advanced the analysis of protein sequences.
• AlphaFold2 revolutionized protein structure prediction, generating vast amounts of 3D structural data.
• Understanding proteins requires integrating their 1D sequence and 3D structural information.

Purpose of the Study:

• To develop a unified model that simultaneously analyzes both protein sequence and structure data.
• To leverage pLMs for modeling the dual nature of proteins as sequences and 3D machines.
• To create a novel approach for translating between 3D structural representations and amino acid sequences.

Main Methods:

• Encoding protein structures into token sequences using the 3Di-alphabet from Foldseek.
• Fine-tuning an existing pLM (ProtT5) on a non-redundant dataset from AlphaFoldDB.
• Developing the Protein 'structure-sequence' T5 (ProstT5) model for joint structure-sequence analysis.

Main Results:

• ProstT5 demonstrated improved performance on structure-related prediction tasks.
• Achieved a three orders of magnitude speedup in deriving 3D structural representations (3Di).
• Established a proof-of-concept for integrating pLMs with large-scale protein structure data.

Conclusions:

• ProstT5 effectively models both protein sequence and structure, bridging the gap between pLMs and structural bioinformatics.
• This approach is crucial for future applications, enabling structure-sensitive searches in large metagenomic databases.
• ProstT5 opens new research avenues in the post-AlphaFold2 era by integrating structural predictions into sequence-based analyses.