Evaluating qualitative assays using sensitivity and specificity

Sensitivity and specificity are two important indices of performance of qualitative assays. Evaluating these indices usually requires one to identify the true disease state of each subject involved in a study. This implies that a perfect test, a "gold standard," is needed to test each subject. However, a gold standard test cannot always be performed on all subjects, whether because of cost or adverse effect on a subject's welfare. In these situations, a common practice is to apply both a currently used assay and an investigational assay to the same specimen. If the testing results are discordant, a gold standard test is applied. This approach has been criticized by many and, in fact, the statistics based on this approach usually overestimate sensitivity and specificity. This paper proposes two alternative methods to estimate sensitivity and specificity. Simulation results show that these methods perform better than the commonly used existing ones. This paper proposes new acceptance criteria and designs to specific topics for the evaluation of blood related assays as well. To evaluate a qualitative assay related to blood specimens, one must also perform studies of storage conditions, interfering substances, and other related factors, in order to establish the equivalency of the assay under standard and various other conditions. To conduct these studies, true negative blood donor specimens are used as a sample from a nondiseased population; and blood donor specimens with spiked analyte are used to represent a sample from a diseased population. Currently, the target-spiking ranges and sample sizes are determined subjectively. This paper presents new acceptance criteria on acceptable conditions and objective standards for selecting the target-spiking range and sample size.