Comparison of performance in successive CASP experiments.

Area of Science:

• Computational biology
• Structural bioinformatics
• Protein structure prediction

Background:

• The Critical Assessment of Protein Structure Prediction (CASP) experiments provide a benchmark for evaluating protein structure prediction methods.
• Standardized performance comparison across successive CASP experiments is essential to track progress and identify areas needing improvement.

Purpose of the Study:

• To analyze and compare performance across CASP1, CASP2, CASP3, and CASP4 using a unified difficulty scale.
• To assess progress in protein structure prediction, particularly in sequence alignment and new fold modeling.

Main Methods:

• Analysis of CASP1-CASP4 results, incorporating CASP4 data into previous analyses.
• Utilization of a unified difficulty scale to compare performance as a function of target difficulty.
• Evaluation of performance in aligning target sequences to structural templates and in new fold modeling.

Main Results:

• Clear improvement in alignment quality between CASP1 and CASP2, with minimal detectable improvement between CASP3 and CASP4.
• Alignment remains the primary source of error for models with less than 30% sequence identity.
• Limited numerical evidence for significant progress in new fold modeling between CASP3 and CASP4, despite subjective consensus on progress.

Conclusions:

• Protein structure prediction has seen slow, incremental progress, particularly in alignment quality.
• Alignment accuracy is a critical bottleneck, especially for targets with low sequence identity.
• Progress in new fold modeling is debated, with subjective agreement on advancement but limited quantitative support.