Evaluation of Cross-Validation Strategies in Sequence-Based Binding Prediction Using Deep Learning.

Area of Science:

• Computational chemistry
• Drug discovery
• Machine learning

Background:

• Deep neural networks (DNNs) show superior performance in predicting drug-target binding compared to traditional methods.
• A key challenge remains the generalization capability of these predictive models.

Purpose of the Study:

• To investigate the impact of various cross-validation strategies and molecular representation methods on the performance of deep learning proteo-chemometrics models for binding prediction.
• To assess the influence of data from different molecular databases and potential biases on model generalization.

Main Methods:

• Evaluated four cross-validation strategies: random splitting, K-means clustering, source database splitting, and a combined approach.
• Applied these strategies to a deep learning proteo-chemometrics model and a logistic regression baseline.
• Tested two molecular descriptors: SMILES strings and molecular fingerprints.

Main Results:

• The deep learning model's performance is comparable to the state-of-the-art.
• Database bias contributes to poor generalization in binding prediction models.
• Restrictive cross-validation, particularly compound clustering, yields less optimal but more robust and credible results.
• Molecular fingerprints provide better model performance than SMILES strings.

Conclusions:

• Generalization issues in binding prediction models stem from biases in public molecular databases.
• Employing stringent cross-validation strategies, like compound clustering, enhances model robustness and credibility, despite potentially lower performance metrics.
• Representing molecules using fingerprints is more effective for building reliable binding prediction models.