MCWS-Transformers: Towards an Efficient Modeling of Protein Sequences via Multi Context-Window Based Scaled...

Area of Science:

• Computational Biology
• Bioinformatics
• Machine Learning

Background:

• Protein sequence modeling is crucial for understanding biological functions.
• Standard transformer networks face limitations in capturing complex patterns in protein sequences.
• Effective representation of protein sequences requires integrating local and global contextual information.

Purpose of the Study:

• To develop an advanced self-attention mechanism for protein sequence modeling.
• To introduce a novel Multi Context-Window based Scaled (MCWS) transformer network.
• To enhance protein function prediction accuracy at the sub-sequence level.

Main Methods:

• A novel context-window based scaled self-attention mechanism was developed.
• This mechanism integrates local context and large contextual patterns.
• The mechanism was implemented within the Multi Context-Window based Scaled (MCWS) transformer network for function prediction.

Main Results:

• The MCWS transformer network demonstrated improved predictive performance over existing methods.
• Significant F1-score improvements of +2.30% (BP) and +2.08% (MF) were achieved compared to standard transformers.
• Outperformed state-of-the-art approaches by +3.38% (BP) and +2.86% (MF), showing robust performance across varying sequence lengths.

Conclusions:

• The proposed MCWS transformer network effectively models protein sequences by leveraging contextual patterns.
• The novel self-attention mechanism enhances the representation of protein sequences.
• This approach offers a substantial advancement in protein function prediction accuracy.