Lack of generalizability of sensitivity and specificity with treatment effects

In biomarker development, two types of summary measures are often used to describe marker accuracy. Positive and negative predictive values describe how well a marker predicts clinical states of interest, while sensitivity and specificity describe how well a marker discriminates between the two states. Insofar as predictive values depend heavily on the prevalence of the clinical states and sensitivity and specificity may not, sensitivity and specificity are preferred in early biomarker development. In many applications, an ideal property of a biomarker is fulfillment of the first Prentice criterion. Under this condition, predictive values do not depend on a covariate (such as treatment) because the biomarker captures all relevant information about the clinical state of interest. A similar condition can be defined for sensitivity and specificity which states that these measures do not depend on a covariate (e.g. treatment). This condition, which we refer to as the equal discriminatory accuracy (EDA) condition, is desirable because it allows sensitivity and specificity from one treatment setting (or covariate value) to be applied to a different setting. We demonstrate, however, that the Prentice condition and EDA are incompatible. Further, under a simple proportional hazards model for a time-to-event outcome, EDA will not be satisfied. We present numerical examples as well as examples of a potential marker in late-stage prostate cancer and another for cervical cancer screening. These results demonstrate that evaluating sensitivity and specificity within treatment (or other covariate) groups is necessary even when simple proportional hazards models or the Prentice criterion holds.