Error Tolerance of Machine Learning Algorithms across Contemporary Biological Targets

Area of Science:

• Computational chemistry
• Machine learning
• Drug discovery

Background:

• Machine learning (ML) models are crucial for predicting drug properties.
• ML model efficacy relies on accurate and abundant data, which are often scarce.
• Investigating data accuracy limitations can reveal alternative data sources for ML.

Purpose of the Study:

• To assess the impact of data inaccuracies on ML model performance in drug development.
• To explore the utility of datasets with known error profiles, such as those from Free Energy Perturbation (FEP+).
• To determine if ML models can be trained on less accurate data for drug property prediction.

Main Methods:

• Introduced varying proportions of error into high-accuracy datasets across diverse targets (kinases, GPCR, polymerases, proteases).
• Evaluated the retrospective accuracy decay of Naïve Bayes Network, Random Forest, and Probabilistic Neural Network models.
• Tested the performance of ML models trained on data simulating FEP+ error profiles.

Main Results:

• Naïve Bayes Networks tolerated up to 39% training set error before losing predictivity.
• Random Forests tolerated an average of 29% training set error.
• Probabilistic Neural Networks showed lower tolerance, losing predictivity at around 20% error.
• Naïve Bayes Networks and Random Forests successfully utilized FEP+-like error profiles.

Conclusions:

• ML models demonstrate notable tolerance to data inaccuracies in drug property prediction.
• Computational methods with known error distributions, like FEP+, can serve as valuable data sources.
• This approach may reduce reliance on extensive and costly in vitro experimental datasets for ML model generation.