Related Experiment Video
Updated: Jun 10, 2026
16:41
A Protocol for Computer-Based Protein Structure and Function Prediction
Published on: November 3, 2011
68.4K
Regularly updated benchmark sets for statistically correct evaluations of AlphaFold applications.
Laszlo Dobson1,2, Gábor E Tusnády1,2, Peter Tompa1,3,4
1Institute of Molecular Life Sciences Research, Centre for Natural Sciences, Magyar Tudósok Körútja, Budapest, Hungary.
Briefings in Bioinformatics
|March 11, 2025
Summary
Scientists often overlook data leakage when using AlphaFold2/3 for protein structure prediction. This study introduces a benchmark set to rigorously evaluate machine learning applications in structural biology.
Area of Science:
- Structural biology
- Computational biology
- Protein science
Background:
- AlphaFold2 significantly advanced protein structure prediction, enabling numerous applications across protein science.
- The widespread adoption of AlphaFold2 has led to optimism, potentially overshadowing critical evaluation of its methods.
Purpose of the Study:
- To address the issue of data leakage in machine learning evaluations for protein structure prediction.
- To provide a rigorous benchmark dataset for assessing AlphaFold2 and AlphaFold3 applications.
Main Methods:
- Development of a novel benchmark dataset designed to detect data leakage.
- Application of the benchmark set to evaluate various protein structure prediction tools and methodologies.
Main Results:
- The benchmark set effectively identifies instances of data leakage in current evaluation practices.
- Demonstration of potential biases introduced by data leakage in AlphaFold-based applications.
Conclusions:
- A rigorous benchmark is essential for reliable evaluation of protein structure prediction tools.
- Addressing data leakage is crucial for advancing the accurate application of AI in structural biology.
Related Concept Videos
01:29Protein Folding Quality Check in the RER
ER is the primary site for the maturation and folding of soluble and transmembrane secretory proteins. The calnexin cycle is a specific chaperone system that folds and assesses the confirmation of N-glycosylated proteins before they can exit the ER lumen. The primary players of this quality check pipeline are the lectins, ER-resident chaperones, and a glucosyl transferase enzyme. In case the calnexin system in the lumen fails to salvage a misfolded protein, it is transported to the cytoplasm...
01:19Expected Frequencies in Goodness-of-Fit Tests
A goodness-of-fit test is conducted to determine whether the observed frequency values are statistically similar to the frequencies expected for the dataset. Suppose the expected frequencies for a dataset are equal such as when predicting the frequency of any number appearing when casting a die. In that case, the expected frequency is the ratio of the total number of observations (n) to the number of categories (k).
01:10Bonferroni Test
The Bonferroni test is a statistical test named after Carlo Emilio Bonferroni, an Italian mathematician best known for Bonferroni inequalities. This statistical test is a type of multiple comparison test to determine which means are different than the rest. Bonferroni test can minimize the Type 1 error by reducing the significance level alpha, which otherwise increases with sample pairs.
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...