SIMPD: an algorithm for generating simulated time splits for validating machine learning approaches.

Area of Science:

• Medicinal Chemistry
• Computational Chemistry
• Machine Learning

Background:

• Time-split cross-validation is the standard for validating predictive models in medicinal chemistry.
• Such data is scarce, particularly outside large pharmaceutical companies.
• Existing data splitting methods do not accurately reflect real-world project data.

Purpose of the Study:

• To introduce the SIMPD (simulated medicinal chemistry project data) algorithm.
• To generate training and test sets that mimic real-world medicinal chemistry project data.
• To create publicly available datasets for machine learning model validation.

Main Methods:

• SIMPD employs a multi-objective genetic algorithm.
• Objectives are based on extensive analysis of compound differences in lead optimization projects.
• The algorithm was applied to ChEMBL bioactivity data.

Main Results:

• SIMPD generates training/test splits that better reflect property and performance differences compared to random or neighbor splits.
• The method accurately mimics differences observed in temporal splits of real-world data.
• 99 public datasets were created using the SIMPD algorithm.

Conclusions:

• SIMPD provides a valuable tool for validating machine learning models in medicinal chemistry.
• The algorithm enables the use of public data for more realistic model assessment.
• The code and datasets are available open-source for broader research use.