识别在公开可用和专有数据集上训练的非目标机器学习模型的性能差异
Aljoša Smajić1, Iris Rami1, Sergey Sosnin1
1Department of Pharmaceutical Sciences, University of Vienna, Josef-Holaubek-Platz 2, 1090 Vienna, Austria.
Chemical research in toxicology
|July 13, 2023
概括
像ChEMBL这样的公开数据库往往会夸大积极结果,扭曲预测毒理学模型. 这项研究揭示了数据偏差如何影响模型性能,并强调了制药研究中需要仔细处理数据的必要性.
科学领域:
- 计算化学和化学信息学
- 毒理学和药物发现
背景情况:
- 公共可用的化合物数据库,如ChEMBL,经常更新新的实验数据.
- 报告的积极实验结果不成比例,导致这些数据库存在偏见.
研究的目的:
- 调查在ChEMBL中对特定非目标的积极和非积极条目的分布.
- 评估这种数据分布对制药行业使用的分类模型性能的影响.
主要方法:
- 对选定的非目标物种在ChEMBL数据库中的复合物条目进行分析.
- 开发和评估使用公共和行业特定数据集的分类模型.
- 对大组化合物的预测空间的可视化,以识别收区域.
主要成果:
- 在公共数据上训练的模型往往会过度预测积极的结果.
- 使用制药行业数据训练的模型比使用公共数据的模型更频繁地预测负面结果.
- 预测空间可视化确定了模型预测趋同的特定化学区域.
结论:
- 公共数据库中固有的偏见显著影响预测毒理学模型的性能.
- 仔细考虑数据来源和分布对于建立可靠的药物发现预测模型至关重要.
- 这些发现支持使用共识建模来预测潜在的不良事件.
相关概念视频
01:19Survival Tree
115
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...
Building a Survival Tree
Constructing a...
115
01:13Sensitivity, Specificity, and Predicted Value
499
In healthcare diagnostics, laboratory tests play a crucial role in identifying and diagnosing a wide range of medical conditions. However, interpreting test results is not always straightforward. An abnormal test result does not always confirm the presence of a disease, just as a normal result does not guarantee its absence. To assess the reliability of these diagnostic tools, healthcare practitioners rely on two key statistical indicators: sensitivity and specificity.
Sensitivity is the...
Sensitivity is the...
499
01:03Prediction Intervals
2.3K
The interval estimate of any variable is known as the prediction interval. It helps decide if a point estimate is dependable.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
2.3K
02:07Improving Translational Accuracy
11.6K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
11.6K
01:52Accuracy and Precision
9.8K
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value. Highly accurate...
9.8K
01:26Pharmacokinetic Models: Comparison and Selection Criterion
109
Physiological and compartmental models are valuable tools used in studying biological systems. These models rely on differential equations to maintain mass balance within the system, ensuring an accurate representation of the dynamic processes at play.
Physiological models take a detailed approach by considering specific molecular processes. They can predict drug distribution, metabolism, and elimination changes, providing a comprehensive understanding of how drugs interact with the body.
Physiological models take a detailed approach by considering specific molecular processes. They can predict drug distribution, metabolism, and elimination changes, providing a comprehensive understanding of how drugs interact with the body.
109