Experimental Errors in QSAR Modeling Sets: What We Can Do and What We Cannot Do.

Area of Science:

• Computational chemistry
• cheminformatics
• Drug discovery

Background:

• Quantitative structure-activity relationship (QSAR) models are crucial for predicting compound activity.
• Data quality significantly impacts QSAR model reliability and predictive performance.
• Experimental errors in chemical data sets can lead to inaccurate QSAR models.

Purpose of the Study:

• To investigate the impact of simulated experimental errors on QSAR model performance.
• To assess the relationship between the ratio of questionable data and QSAR modeling outcomes.
• To determine if removing compounds with high cross-validation errors improves external prediction accuracy.

Main Methods:

• Eight curated data sets (four continuous, four categorical endpoints) were used.
• Simulated experimental errors were introduced by randomizing compound activities.
• Over 1800 QSAR models were built and evaluated using fivefold cross-validation.
• Models were tested on external, un-modeled compound sets.

Main Results:

• QSAR model performance significantly deteriorated as the ratio of simulated experimental errors increased.
• Compounds with large cross-validation prediction errors often corresponded to those with simulated errors.
• Removing compounds with high cross-validation errors did not enhance the prediction of external data sets.
• Overfitting was observed when attempting to improve model predictivity by removing problematic compounds.

Conclusions:

• QSAR predictions, particularly consensus predictions, can effectively identify compounds with potential experimental errors.
• Relying solely on cross-validation to remove erroneous data is not a reliable strategy for improving QSAR model predictivity.
• Careful data curation and validation remain essential for robust QSAR modeling.