Related Experiment Video
Updated: Apr 4, 2026
Protein WISDOM: A Workbench for In silico De novo Design of BioMolecules
Published on: July 25, 2013
A Web Resource for Standardized Benchmark Datasets, Metrics, and Rosetta Protocols for Macromolecular Modeling and
Shane Ó Conchúir1, Kyle A Barlow2, Roland A Pache1
1California Institute for Quantitative Biosciences (QB3), University of California San Francisco, San Francisco, California, United States of America; Department of Bioengineering and Therapeutic Sciences, University of California San Francisco, San Francisco, California, United States of America.
A new web resource offers benchmark datasets and metrics for computational protein modeling and design. This resource guides protocol development and comparison, aiming to improve accuracy and adoption in biological applications.
Area of Science:
- Computational Biology
- Biophysics
- Structural Biology
Background:
- Development and validation of computational macromolecular modeling and design methods require benchmark datasets and performance metrics.
- Existing resources for established disciplines provide expert implementations for comparison.
- Broader adoption of methods necessitates information on appropriate parameters and accuracy compared to experimental data.
Purpose of the Study:
- To establish a freely accessible web resource for computational macromolecular modeling and design protocols.
- To guide the development and validation of new modeling and design methods.
- To facilitate the comparison of different computational protocols and identify best practices.
Main Methods:
- Presentation of a web resource (https://kortemmelab.ucsf.edu/benchmarks) with benchmark datasets and metrics.
- Inclusion of downloadable benchmark capture archives with input files, analysis scripts, and tutorials.
- Provision of command lines for running benchmarks using the Rosetta software suite.
Main Results:
- The resource provides datasets and metrics for comparing various modeling protocols, including different computational sampling methods and energy functions.
- Initial benchmarks cover prediction of energetic effects of mutations, protein design, and protein structure prediction.
- Each benchmark includes "best practice" parameters and associated state-of-the-art modeling protocols.
Conclusions:
- The developed web resource serves as a valuable tool for the macromolecular modeling and design community.
- It facilitates the standardization and improvement of computational protocols.
- Future expansion of benchmarks and evaluation of new methods are anticipated with community involvement.
