Step Forward Cross Validation for Bioactivity Prediction: Out of Distribution Validation in Drug Discovery.

Area of Science:

• Computational chemistry
• Medicinal chemistry
• Machine learning

Background:

• Machine learning (ML) advances in materials science offer accurate property predictions.
• Adapting ML for drug discovery requires addressing prospective validation for out-of-distribution (OOD) data.
• Assessing model performance on OOD data is crucial for real-world applications.

Purpose of the Study:

• To evaluate k-fold n-step forward cross-validation for OOD small molecule bioactivity prediction.
• To assess the utility of discovery yield and novelty error metrics in drug discovery models.
• To enhance the accuracy and applicability of ML models in drug discovery.

Main Methods:

• Implemented k-fold n-step forward cross-validation for bioactivity prediction.
• Compared k-fold n-step forward cross-validation with conventional random split cross-validation.
• Analyzed discovery yield and novelty error to evaluate model performance and applicability domain.

Main Results:

• K-fold n-step forward cross-validation demonstrated improved accuracy for OOD bioactivity predictions compared to random splits.
• This cross-validation method better reflects real-world drug discovery model performance.
• Discovery yield and novelty error metrics provided insights into model applicability and predictive power for desirable bioactivity.

Conclusions:

• K-fold n-step forward cross-validation is more effective than random splits for OOD bioactivity prediction.
• Discovery yield and novelty error are valuable metrics for assessing drug discovery models.
• Recommend integrating k-fold n-step forward cross-validation and these metrics for state-of-the-art bioactivity prediction models.