Predicting enzymatic function of protein sequences with attention.

Area of Science:

• Bioinformatics
• Computational Biology
• Machine Learning

Background:

• The vast majority of protein sequences lack manual functional annotation.
• Automated methods are crucial to bridge the gap between sequence availability and functional understanding.
• Predicting enzymatic classes from sequence alone is a key challenge in functional annotation.

Purpose of the Study:

• To investigate the efficacy of Transformer deep neural networks for predicting enzymatic classes from protein sequences.
• To develop and evaluate EnzBert, a specialized protein language model for Enzyme Commission (EC) number prediction.
• To compare EnzBert's performance against state-of-the-art methods for enzyme functional annotation.

Main Methods:

• Utilized Transformer deep neural networks, specifically EnzBert, a specialized protein language model.
• Trained models to predict Enzyme Commission (EC) numbers at different hierarchical levels.
• Developed new time-based benchmarks for evaluating prediction accuracy at the most detailed EC level.
• Employed attention maps for model interpretability and identification of important residues.

Main Results:

• EnzBert achieved 95% accuracy in predicting EC numbers at level two, outperforming previous tools (84%).
• Significantly improved macro-F1 scores at the detailed EC level four (54% vs. 41% and 26% vs. 20%).
• Attention maps effectively identified key residues, correlating with known catalytic sites, achieving a max F-Gain score of 96.05%.

Conclusions:

• Transformer models like EnzBert offer a powerful approach for automated protein functional annotation, particularly for enzymatic classes.
• EnzBert demonstrates superior performance compared to existing methods for EC number prediction.
• Attention-based interpretability methods provide insights into model predictions and biological relevance.