ProteinGLUE multi-task benchmark suite for self-supervised protein modeling.

Area of Science:

• Computational biology
• Bioinformatics
• Machine learning in biology

Background:

• Self-supervised language modeling is advancing protein sequence analysis.
• Current methods lack standardization, hindering model comparison and generalizability.
• Existing evaluations often focus on limited downstream tasks.

Purpose of the Study:

• Introduce ProteinGLUE, a standardized benchmark for evaluating protein representations.
• Facilitate rigorous comparison of diverse self-supervised learning models for protein sequences.
• Provide baseline models and code to accelerate research in protein property prediction.

Main Methods:

• Developed ProteinGLUE, a benchmark comprising seven per-amino-acid tasks.
• Implemented two baseline models trained on masked symbol and next sentence prediction.
• Evaluated model performance on downstream tasks including secondary structure and interface prediction.

Main Results:

• Pre-training significantly improved performance on various downstream tasks compared to no pre-training.
• The larger base model did not consistently outperform the smaller medium model.
• ProteinGLUE provides a robust framework for assessing learned protein representations.

Conclusions:

• ProteinGLUE offers a valuable resource for the protein sequence analysis community.
• Standardized evaluation is crucial for advancing self-supervised learning in proteomics.
• The benchmark facilitates the development of more generalizable and effective protein representation models.