Bag-of-words is competitive with sum-of-embeddings language-inspired representations on protein inference

Area of Science:

• Computational Biology
• Bioinformatics
• Genomics

Background:

• Protein function inference is crucial for understanding biological mechanisms but laboratory experiments are costly.
• Computational methods using amino acid sequences are needed for large-scale protein function prediction.
• Natural language processing and self-supervised learning have recently been applied to derive features from protein sequences.

Purpose of the Study:

• To evaluate the effectiveness of self-supervised learning-based protein representations for function inference tasks.
• To compare these representations against simpler baseline methods like bag-of-words histograms.
• To identify key features that contribute to accurate data-driven protein function prediction.

Main Methods:

• Utilized self-supervised pre-training on a large protein sequence database to learn representations.
• Applied these learned representations to various protein inference tasks, including sequence similarity.
• Compared performance against bag-of-words histogram representations.
• Employed feature selection techniques to identify important discriminative features.

Main Results:

• Bag-of-words histogram representations demonstrated superior performance compared to self-supervised learning-based representations on sequence similarity and protein inference tasks.
• Feature selection identified specific discriminant features that enhance the predictive power of bag-of-words models.
• The study highlights limitations of current self-supervised approaches for capturing essential biological information from sequence alone.

Conclusions:

• Simple bag-of-words models are currently more effective than self-supervised learning for protein function inference based on sequence data.
• Feature selection is a valuable strategy for improving the performance of bag-of-words models in this domain.
• Further research into alternative pre-training schemes is encouraged to develop self-supervised models that better capture biologically relevant information from protein sequences.