Assessing the reliability of a QSAR model's predictions

Quantitative structure activity relationships (QSAR) are one of the well-developed areas in computational chemistry. In this field, many successful predictive models have been developed for various property, activity or toxicity predictions. However, the predictive power of models for new query compounds is often not well characterized. The breadth of applicability of models is often not characterized. In other words, with a given QSAR model and a specific query compound to be predicted, can the model be used reliably for the desired prediction? In this study, we assessed the reliability of QSAR models' prediction on query compounds. Our approach, employing hierarchical clustering, was developed and tested using a test dataset containing 322 organic compounds with fathead minnow acute aquatic toxicity as the activity of interest. The hypothesis of the approach was that if a query compound is more similar to the compounds used to generate the QSAR model, it should be predicted more accurately. Thus, the core of the approach is to determine the relationship between the similarity of query compounds to the training set compounds of the QSAR model and the prediction accuracy given by that model. This relationship determination was achieved by comparing the results given by the two major components of the approach: objects clustering and activity prediction. With the resultant information from the two steps, a direct relationship was shown.