A fast (CNN + MCWS-transformer) based architecture for protein function prediction.

Area of Science:

• Bioinformatics
• Computational Biology
• Machine Learning

Background:

• Transformer models have revolutionized biological sequence mining but face computational challenges due to quadratic complexity.
• High complexity (O(l^2)) in transformers limits training and prediction efficiency for long biological sequences.

Purpose of the Study:

• To develop a simplified, generalized, and efficient transformer architecture for enhanced protein function prediction (PFP).
• To address the computational limitations of standard transformers in biological sequence analysis.

Main Methods:

• A novel transformer architecture combining Convolutional Neural Networks (CNN) and global-average pooling to shorten protein sequences.
• Implementation of focal loss for balanced training, particularly for challenging classifications.
• Development of a multi sub-sequence-based PFP solution using an average-pooling layer with a stride of 2.

Main Results:

• The proposed architecture reduces transformer complexity to O((l/2)^2), significantly speeding up computation.
• Achieved performance improvements of +2.50% (Biological Process - BP) and +3.00% (Molecular Function - MF) over Global-ProtEnc Plus.
• Demonstrated superior performance over Lite-SeqCNN with improvements of +4.50% (BP) and +2.30% (MF).

Conclusions:

• The developed transformer architecture offers a computationally efficient and accurate solution for protein function prediction.
• The combination of sequence shortening techniques and focal loss effectively enhances model performance on biological sequence data.
• This work presents a significant advancement in applying deep learning for biological sequence mining and function prediction.