Accuracy of protein-level disorder predictions

Area of Science:

• Biochemistry and Molecular Biology
• Computational Biology
• Structural Biology

Background:

• Experimental characterization of intrinsic protein disorder is limited, covering only a small fraction of sequenced proteins.
• Over 60 sequence-based disorder predictors have been developed to address this gap.
• Current benchmarks assess predictor performance on protein datasets, not individual protein predictions.

Purpose of the Study:

• To perform a protein-level assessment of 13 popular sequence-based disorder predictors.
• To evaluate the variation in predictive performance across individual proteins.
• To identify limitations and needs for future development of disorder prediction tools.

Main Methods:

• A protein-level assessment was conducted using 6,200 disorder-annotated proteins.
• The performance of 13 popular disorder predictors was evaluated on an individual protein basis.
• Comparison of protein-level performance distributions against traditional dataset-level benchmarks.

Main Results:

• Protein-level performance is generally higher than dataset-level benchmarks, with 57-75% of proteins showing improved prediction quality.
• A significant portion (up to 30%) of proteins, particularly those with high disorder, exhibit poor predictive performance.
• Individual predictors excel for specific proteins, and most proteins are accurately predicted by multiple tools, yet challenges remain for hard-to-predict proteins.

Conclusions:

• Current disorder predictors generally outperform dataset-level benchmarks for the majority of proteins and show complementary strengths.
• There is a critical need for novel computational tools capable of accurately predicting disorder in challenging proteins.
• Further research should focus on improving the accuracy and reliability of predictions for proteins with substantial intrinsic disorder.