The ability of short-term tests to predict carcinogenicity can be summarized in a single index

This paper presents a new procedure aimed at quantifying the ability of short-term tests (STT) to discriminate between carcinogens and noncarcinogens. While the sensitivity, specificity, and accuracy indices provide an estimate that is not biased by the relative proportion of carcinogens and noncarcinogens of the different databases, the new index, called Relative Operating Characteristics (ROC), goes a step further, and overcomes the problem of the decision criterion bias. In fact, if the threshold--that is, minimum increase over the control above which an STT result is positive--is varied, sensitivity and other factors are also consequently affected. A similar problem occurs when the STTs are assembled in batteries: the performance of the battery in discriminating between carcinogens and noncarcinogens depends on the preliminary decision of how many and which assays should be positive for considering a chemical as positive in the battery. The ROC analysis produces a single value, which places the performance of different systems on a common, easily interpreted scale (instead of using several different indices such as sensitivity, etc.). Most importantly, this index is independent of the decision criterion bias; consequently it is the best measure of the true ability of STTs to discriminate between carcinogens and noncarcinogens. To illustrate the approach, the ROC analysis is applied to a battery composed of the four in vitro assays studied by the U.S. National Toxicology Program: the analysis confirmed the previous results, pointing to the limited ability of this battery to discriminate between carcinogens and noncarcinogens.