Prediction Accuracy of Production ADMET Models as a Function of Version: Activity Cliffs Rule

Area of Science:

• Computational Chemistry
• Drug Discovery
• Pharmacokinetics and Toxicology

Background:

• Quantitative structure-activity relationship (QSAR) models are crucial for predicting absorption, distribution, metabolism, excretion, and toxicity (ADMET) endpoints.
• Regular model updates are standard practice in many institutions to maintain predictive power.

Purpose of the Study:

• To evaluate the version-to-version predictivity of QSAR models for ADMET endpoints over a 10-year period.
• To investigate the reasons behind unexpected variations in model performance, particularly concerning predictive accuracy.

Main Methods:

• Monitoring the predictive accuracy of new molecules against existing model versions (V) before updating to the next version (V+1).
• Utilizing cell-based permeability (Papp) assay data as a case study for QSAR model analysis.
• Analyzing root-mean-square-error (RMSE) to quantify model predictivity variations.

Main Results:

• QSAR models for ADMET endpoints, including Papp, are generally predictive and useful for chemical design.
• Unexpected and significant variations in model predictivity (RMSE) were observed across different model versions.
• The primary driver for decreased prediction accuracy was identified as "activity cliffs"—discrepancies between predicted and observed activities for new molecules relative to the training set.

Conclusions:

• While QSAR models are valuable tools in drug discovery, their performance can be unpredictable.
• "Activity cliffs" represent a significant challenge, leading to reduced accuracy when new chemical entities deviate from established structure-activity relationships.
• Understanding and addressing activity cliffs is essential for reliable ADMET prediction and effective chemical design.