On the Misleading Use of for QSAR Model Comparison

Area of Science:

• * Computational chemistry
• * Cheminformatics
• * Predictive modeling

Background:

• * Quantitative Structure-Activity Relationship (QSAR) models are crucial in drug discovery, toxicology, and regulatory science.
• * Accurate assessment of QSAR model predictive ability is essential for reliable application.
• * Numerous metrics for evaluating QSAR predictive ability have led to confusion in model comparison.

Purpose of the Study:

• * To clarify the appropriate and inappropriate applications of the $$ metric for QSAR model evaluation.
• * To highlight the limitations of the $$ metric when comparing models trained on different datasets.
• * To guide researchers on selecting suitable metrics for robust QSAR model assessment.

Main Methods:

• * Analysis of the behavior of the $$ metric concerning training data distribution.
• * Illustration of scenarios where $$ estimates can be misleading.
• * Comparison of $$ with dispersion measures like root-mean-square error (RMSE).

Main Results:

• * The $$ metric is well-suited for comparing external predictivity of models trained on the *same* dataset.
• * The $$ metric is inadequate and potentially misleading when comparing models trained on *different* datasets.
• * Dispersion measures, such as RMSE, are recommended for comparing models trained on different datasets.

Conclusions:

• * Proper selection of evaluation metrics is critical for reliable QSAR model comparison.
• * Researchers should utilize dispersion measures (e.g., RMSE) when evaluating QSAR models developed on distinct training datasets.
• * Avoid using the $$ metric for cross-dataset QSAR model comparisons to prevent misleading conclusions.