Experimental Error, Kurtosis, Activity Cliffs, and Methodology: What Limits the Predictivity of Quantitative...

Area of Science:

• Medicinal Chemistry
• Computational Chemistry
• Cheminformatics

Background:

• Quantitative structure-activity relationship (QSAR) model predictivity varies significantly across datasets.
• Recent research suggests data limitations, specifically activity cliffs, rather than QSAR methodology, are the primary constraint on modelability.
• Activity cliffs represent small structural changes leading to large activity changes, posing challenges for predictive modeling.

Purpose of the Study:

• To investigate the predictive power of experimental error, activity distribution, and activity cliff metrics in determining QSAR dataset predictivity.
• To evaluate the effectiveness of these metrics across diverse in-house datasets, including those with manipulated activity distributions and added noise.
• To assess the performance of modern QSAR methods and alternative descriptors when dealing with activity cliffs.

Main Methods:

• Analysis of in-house QSAR datasets with varying characteristics (unmodified, structurally determined, manipulated activity distributions, added noise).
• Calculation and comparison of experimental error, activity distribution metrics, and activity cliff metrics.
• Evaluation of QSAR model performance using random-split cross-validation and time-split validation.
• Assessment of modern QSAR methods and alternative descriptors on activity cliff compounds.

Main Results:

• Activity cliff metrics were superior predictors of dataset predictivity compared to experimental error and activity distribution metrics across all dataset types.
• Activity cliff metrics struggled to differentiate real cliffs caused by high uncertainty in activity measurements.
• Modern QSAR methods and alternative descriptors showed limited success in predicting activities for compounds on activity cliffs, aligning with modelability assumptions.
• Time-split predictivity was related to random-split predictivity, with chemical space coverage being as critical as activity uncertainty and activity cliffs.

Conclusions:

• Activity cliff metrics are essential for evaluating QSAR dataset modelability, outperforming other investigated metrics.
• High uncertainty in activity measurements can obscure true activity cliffs, complicating their identification.
• The limitations of current QSAR methodologies and descriptors in handling activity cliffs persist.
• Both chemical space coverage and data quality (activity uncertainty and cliff presence) are critical factors limiting QSAR predictivity.