定期更新的基准数据集,用于对AlphaFold应用程序进行统计上正确的评估
Laszlo Dobson1,2, Gábor E Tusnády1,2, Peter Tompa1,3,4
1Institute of Molecular Life Sciences Research, Centre for Natural Sciences, Magyar Tudósok Körútja, Budapest, Hungary.
Briefings in bioinformatics
|March 11, 2025
概括
科学家们经常在使用AlphaFold2/3来预测蛋白质结构时忽视数据泄露. 本研究引入了一个基准,用于严格评估结构生物学中的机器学习应用.
科学领域:
- 结构生物学是结构生物学.
- 计算生物学是一种计算生物学.
- 蛋白质科学是一种蛋白质科学.
背景情况:
- AlphaFold2显著提升了蛋白质结构预测,使蛋白质科学中的许多应用成为可能.
- AlphaFold2的广泛采用导致了乐观,可能会掩盖对其方法的批判性评估.
研究的目的:
- 为了解决机器学习评估中数据泄露的问题,用于蛋白质结构预测.
- 为评估AlphaFold2和AlphaFold3应用提供严格的基准数据集.
主要方法:
- 开发一个新的基准数据集,旨在检测数据泄露.
- 应用基准集来评估各种蛋白质结构预测工具和方法.
主要成果:
- 该基准集有效地识别了当前评估实践中数据泄露的实例.
- 在基于AlphaFold的应用程序中,通过数据泄露引入的潜在偏差的演示.
结论:
- 严格的基准对于可靠评估蛋白质结构预测工具至关重要.
- 解决数据泄漏对于推动人工智能在结构生物学中的准确应用至关重要.
相关概念视频
Protein Folding Quality Check in the RER
ER is the primary site for the maturation and folding of soluble and transmembrane secretory proteins. The calnexin cycle is a specific chaperone system that folds and assesses the confirmation of N-glycosylated proteins before they can exit the ER lumen. The primary players of this quality check pipeline are the lectins, ER-resident chaperones, and a glucosyl transferase enzyme. In case the calnexin system in the lumen fails to salvage a misfolded protein, it is transported to the cytoplasm...
Expected Frequencies in Goodness-of-Fit Tests
A goodness-of-fit test is conducted to determine whether the observed frequency values are statistically similar to the frequencies expected for the dataset. Suppose the expected frequencies for a dataset are equal such as when predicting the frequency of any number appearing when casting a die. In that case, the expected frequency is the ratio of the total number of observations (n) to the number of categories (k).
Bonferroni Test
The Bonferroni test is a statistical test named after Carlo Emilio Bonferroni, an Italian mathematician best known for Bonferroni inequalities. This statistical test is a type of multiple comparison test to determine which means are different than the rest. Bonferroni test can minimize the Type 1 error by reducing the significance level alpha, which otherwise increases with sample pairs.
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...


