Data Exploration for Target Predictions Using Proprietary and Publicly Available Data Sets.

Area of Science:

• Computational chemistry
• Drug discovery
• Machine learning in pharmacology

Background:

• Machine learning (ML) models for bioactivity prediction often integrate data from various assay sources.
• Differences in data domains and sources can lead to high variance in bioactivity values and distinct chemical space coverage.
• The origin of training data significantly influences the effectiveness and applicability domain of ML prediction models.

Purpose of the Study:

• To investigate the chemical space and active/inactive compound distribution of proprietary pharmaceutical data (Bayer AG) versus public data (ChEMBL).
• To assess the impact of these data sources on the performance of ML classification models.
• To explore strategies for creating robust mixed training datasets.

Main Methods:

• Applied two descriptor sets and various ML algorithms to analyze Bayer AG and ChEMBL datasets.
• Evaluated prediction performance using Matthews Correlation Coefficient (MCC) values.
• Assessed chemical space overlap using mean Tanimoto similarity of nearest neighbors.
• Investigated mixed training data strategies incorporating assay format and Tanimoto similarity.

Main Results:

• Substantial differences in chemical space were observed between Bayer AG and ChEMBL datasets.
• Models trained on one dataset showed suboptimal performance when tested on the other (MCC values between -0.34 and 0.37).
• Low mean Tanimoto similarity (≤0.3) indicated limited overlap in chemical space for many targets.
• Methods assessing chemical space overlap did not reliably predict model performance across datasets.

Conclusions:

• Data source heterogeneity significantly impacts ML model generalizability in bioactivity prediction.
• Proprietary and public datasets often represent distinct chemical spaces, limiting cross-dataset model applicability.
• Developing effective strategies for integrating diverse data sources, potentially using assay information, is crucial for improving model robustness.