Why QSAR fails: an empirical evaluation using conventional computational approach

Although a number of pitfalls of QSAR have been corrected in the past decade, the reliability of QSAR models is still insufficient. The reason why QSAR fails is still under hot debate; our study attempts to address this topic from a practical and empirical perspective, evaluating two relatively large toxicological data sets using a typical combination of support vector machine (SVM) and genetic algorithm (GA). Our results suggest that the vast number of equivalent models to be chosen and the insufficient validation strategy are primarily responsible for the failure of many QSAR models. First, a method often produces much more equivalent models than we might expect, and the corresponding descriptor sets show little overlap, indicating the unreliability of the conventional approaches. Moreover, although external validation has been considered necessary, validation on an arbitrarily selected independent set is still insufficient to guarantee the true predictability of a QSAR model. Therefore, more effective training and validation strategies are demanded to enhance the reliability of QSAR models. The present study also demonstrates that combinatorial or ensemble models can greatly reduce the variance of equivalent models, and that models built with the most frequently selected descriptors used by the equivalent models seem to yield more promising performances.