A Benchmark Dataset for Evaluating Practical Performance of Model Quality Assessment of Homology Models

Area of Science:

• Structural Bioinformatics
• Computational Biology
• Protein Science

Background:

• Protein structure prediction is crucial in structural bioinformatics.
• Model quality assessment (MQA) is vital for estimating predicted protein structure accuracy.
• Current MQA evaluation relies heavily on the Critical Assessment of Protein Structure Prediction (CASP) dataset.

Purpose of the Study:

• To address limitations of the CASP dataset for MQA evaluation, particularly its lack of high-quality models and inclusion of de novo methods.
• To create a new benchmark dataset, the Homology Models Dataset for Model Quality Assessment (HMDM), specifically for evaluating MQA on homology models.
• To benchmark and compare the performance of various MQA methods using the novel HMDM datasets.

Main Methods:

• Development of the Homology Models Dataset for Model Quality Assessment (HMDM) containing high-quality homology models.
• Benchmarking of existing Model Quality Assessment (MQA) methods using the HMDM datasets.
• Comparison of MQA performance against classical template sequence identity-based selection and statistical potentials.

Main Results:

• The newly created HMDM datasets enable effective evaluation of MQA performance on high-accuracy homology models.
• Latest deep learning-based Model Quality Assessment (MQA) methods demonstrate superior performance compared to template sequence identity.
• Deep learning MQA methods also outperform traditional statistical potentials in selecting accurate protein models.

Conclusions:

• The HMDM dataset provides a valuable resource for assessing Model Quality Assessment (MQA) methods, especially for homology modeling.
• Deep learning approaches represent the state-of-the-art in Model Quality Assessment (MQA), surpassing conventional methods.
• Accurate MQA is essential for advancing protein structure prediction and its applications in structural bioinformatics.