Improving Protein Secondary Structure Prediction by Deep Language Models and Transformer Networks

Area of Science:

• Computational biology
• Bioinformatics
• Structural biology

Background:

• Protein secondary structure prediction is crucial for understanding protein function and is often framed as a sequence-to-sequence translation problem.
• Traditional methods rely on pre-computed statistical profiles derived from multiple sequence alignments (MSAs).

Purpose of the Study:

• To develop a novel, high-performance protein secondary structure prediction tool named TransPross.
• To leverage natural language processing techniques, specifically transformer networks and attention mechanisms, for improved prediction accuracy.

Main Methods:

• TransPross utilizes transformer networks and attention mechanisms to directly extract evolutionary information from raw multiple sequence alignments (MSAs).
• The attention mechanism effectively captures long-range residue-residue interactions within protein sequences.

Main Results:

• TransPross demonstrates superior performance compared to state-of-the-art methods across multiple benchmark datasets.
• Prediction accuracy, measured by the Q3 score, exceeds 80% for difficult targets with limited homologous sequences in their MSAs.
• Accuracy correlates positively with the depth of the MSAs used.

Conclusions:

• TransPross offers a significant advancement in protein secondary structure prediction by directly utilizing evolutionary information via advanced deep learning techniques.
• The tool shows particular promise for predicting structures of proteins with sparse evolutionary data.
• TransPross is publicly available for research use.