Balancing Data Quantity and Quality: Evaluating Curation Strategies for Bioactivity Prediction in Lead Optimization

Building good machine-learning (ML) models to predict the bioactivity of novel chemical matter remains a challenging task. Accurate models require a training set with a large number of diverse compounds and a low level of noise. When extracting data from public databases such as ChEMBL, different levels of curation rigor may be applied, resulting in training sets of varying size, diversity, and, presumably, noise levels. It is not possible to know a priori whether increasing the size of the data set at the cost of adding more noise improves model generalization. To assess this trade-off, we compare three data curation and modeling approaches: (1) models trained on data for a single target, (2) models trained on target-specific data further restricted to a single set of assay conditions, and (3) multitask learning (MTL) models where each assay condition is treated as a separate task. This MTL approach was designed to bridge the gap between data quantity and quality. Graph neural networks (GNN) and random forests (RF) regressors are evaluated via a leave-assay-out strategy to minimize noise in the test sets. Our results show no meaningful performance differences between these curation strategies, suggesting that for lead-optimization tasks, increasing data quantity at the expense of label consistency does not improve generalization. Notably, the MTL approach also failed to provide a performance advantage. Additionally, we find that GNNs exhibit high seed-dependent variability in connection with the comparatively small training sets common for bioactivity measurements, highlighting the necessity of multiseed evaluation for robust model assessment.