注意通过超参数优化进行过拟合!
Igor V Tetko1,2, Ruud van Deursen3, Guillaume Godin4
1Institute of Structural Biology, Molecular Targets and Therapeutics Center, Helmholtz Munich - Deutsches Forschungszentrum Für Gesundheit Und Umwelt (GmbH), 86764, Neuherberg, Germany. igor.tetko@helmholtz-munich.de.
Journal of cheminformatics
|December 9, 2024
概括
机器学习中的超参数优化可能会导致过拟合. 使用预设的超参数提供了类似的结果,大大减少了计算时间,并提高了模型准确性.
科学领域:
- 计算化学的计算化学
- 机器学习 机器学习
- 药物发现 药物发现 药物发现
背景情况:
- 超参数优化在机器学习中常见,用于诸如可溶性预测等任务.
- 之前的研究使用了基于图的方法,对各种可溶性数据集进行了分析.
- 人们担心在广泛的超参数调整过程中可能会出现过度装配.
研究的目的:
- 研究超参数优化对溶解性预测中的模型性能的影响.
- 为了比较预设的超参数与优化的超参数的效率和准确性.
- 为了评估一种新的基于自然语言处理 (Natural Language Processing) 的表示学习方法,Transformer CNN.
主要方法:
- 对七个热力学和动力学可溶性数据集的分析.
- 基于图形的最新方法与超参数优化和预设超参数的比较.
- 实现和评估变换器CNN,一种使用SMILES字符串的自然语言处理方法.
主要成果:
- 超参数优化并没有持续改善模型性能,可能导致过拟合.
- 具有预设超参数的模型实现了与优化模型相比的结果,将计算成本降低了大约1万倍.
- 变压器CNN在28次比较中26次超越了基于图表的方法,证明了卓越的准确性和效率.
结论:
- 预先优化的超参数可能会因为过度拟合而对模型概括产生负面影响.
- 使用预设的超参数是一种计算效率高的策略,可以产生可比的预测性能.
- 变压器CNN代表了可溶性预测准确性和速度的重大进步.
相关概念视频
01:16Goodness-of-Fit Test
3.3K
The goodness-of-fit test is a type of hypothesis test which determines whether the data "fits" a particular distribution. For example, one may suspect that some anonymous data may fit a binomial distribution. A chi-square test (meaning the distribution for the hypothesis test is chi-square) can be used to determine if there is a fit. The null and alternative hypotheses may be written in sentences or stated as equations or inequalities. The test statistic for a goodness-of-fit test is given as...
3.3K
01:19Expected Frequencies in Goodness-of-Fit Tests
2.5K
A goodness-of-fit test is conducted to determine whether the observed frequency values are statistically similar to the frequencies expected for the dataset. Suppose the expected frequencies for a dataset are equal such as when predicting the frequency of any number appearing when casting a die. In that case, the expected frequency is the ratio of the total number of observations (n) to the number of categories (k).
2.5K
01:52Regression Toward the Mean
6.3K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.3K
02:07Improving Translational Accuracy
9.0K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
9.0K
01:25Multiple Regression
2.9K
Multiple regression assesses a linear relationship between one response or dependent variable and two or more independent variables. It has many practical applications.
Farmers can use multiple regression to determine the crop yield based on more than one factor, such as water availability, fertilizer, soil properties, etc. Here, the crop yield is the response or dependent variable as it depends on the other independent variables. The analysis requires the construction of a scatter plot...
Farmers can use multiple regression to determine the crop yield based on more than one factor, such as water availability, fertilizer, soil properties, etc. Here, the crop yield is the response or dependent variable as it depends on the other independent variables. The analysis requires the construction of a scatter plot...
2.9K
01:03Prediction Intervals
2.2K
The interval estimate of any variable is known as the prediction interval. It helps decide if a point estimate is dependable.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
2.2K