Evaluation of template-based modeling in CASP13.

Area of Science:

• Computational biology
• Structural bioinformatics
• Protein structure prediction

Background:

• The Critical Assessment of protein Structure Prediction (CASP) is a community-wide experiment to assess protein structure prediction methods.
• Template-based modeling (TBM) is a key category in CASP, relying on known protein structures.

Purpose of the Study:

• To assess the performance of template-based modeling groups in CASP13.
• To introduce and evaluate novel metrics for assessing protein model quality beyond traditional CASP metrics.

Main Methods:

• Ranking predictor groups using CASP12 primary ranking score.
• Analyzing performance based on template availability (TBM-easy vs. TBM-hard).
• Evaluating models using traditional CASP metrics and new metrics (torsion angles, molecular replacement utility).

Main Results:

• Top performance achieved by groups using deep multiple sequence alignments for contact or distance predictions.
• Modifying existing templates yields best results for TBM-easy cases.
• Deep learning algorithms trained on the Protein Data Bank (wwPDB) achieve excellent results for TBM-hard cases without explicit templates.
• New metrics show poor correlation with traditional metrics, indicating limitations of current assessment.

Conclusions:

• Deep learning approaches are highly effective, particularly for challenging protein structure prediction targets.
• Protein modeling has matured, necessitating broader evaluation criteria beyond traditional metrics.
• Future assessments should incorporate alternative metrics to capture model utility and accuracy more comprehensively.