Interlaboratory evaluation of genomic signatures for predicting carcinogenicity in the rat

The Critical Path Institute recently established the Predictive Safety Testing Consortium, a collaboration between several companies and the U.S. Food and Drug Administration, aimed at evaluating and qualifying biomarkers for a variety of toxicological endpoints. The Carcinogenicity Working Group of the Predictive Safety Testing Consortium has concentrated on sharing data to test the predictivity of two published hepatic gene expression signatures, including the signature by Fielden et al. (2007, Toxicol. Sci. 99, 90-100) for predicting nongenotoxic hepatocarcinogens, and the signature by Nie et al. (2006, Mol. Carcinog. 45, 914-933) for predicting nongenotoxic carcinogens. Although not a rigorous prospective validation exercise, the consortium approach created an opportunity to perform a meta-analysis to evaluate microarray data from short-term rat studies on over 150 compounds. Despite significant differences in study designs and microarray platforms between laboratories, the signatures proved to be relatively robust and more accurate than expected by chance. The accuracy of the Fielden et al. signature was between 63 and 69%, whereas the accuracy of the Nie et al. signature was between 55 and 64%. As expected, the predictivity was reduced relative to internal validation estimates reported under identical test conditions. Although the signatures were not deemed suitable for use in regulatory decision making, they were deemed worthwhile in the early assessment of drugs to aid decision making in drug development. These results have prompted additional efforts to rederive and evaluate a QPCR-based signature using these samples. When combined with a standardized test procedure and prospective interlaboratory validation, the accuracy and potential utility in preclinical applications can be ascertained.