Methodological aspects of the evaluation of postoperative cancer surveillance. Part I: Validity

This contribution deals with problems of determining the validity and efficacy of diagnostic procedures used in postoperative cancer surveillance. While there is a large number of follow-up studies attempting to estimate sensitivity, specificity, or the predictive values of follow-up tests, the methodological aspects of validation are rarely addressed adequately in these publications, and the estimations of validity are often not well-founded. In the first part of this paper we analyze aspects of validity and validation studies. In particular, the following problems are addressed: a) Validity depends on the context; in particular, it depends on the characteristics of the population, the embedding of the test in a diagnostic strategy, the uncertainty of the test itself, the gold standard, and the criterion for positive test results. b) Global, time-independent measures of validity are hardly adequate. In particular, any attempt to incorporate a positive lead-time in the definition of the sensitivity is incompatible with an exhaustive classification matrix and with a meaningful definition of predictive values. c) Published estimations of sensitivity mostly suffer from numerous biases. First, there is work-up bias, which arises if positive and negative test-results are verified in a different way (in fact, this implies the use of two different gold standards). Next, test-review bias may occur unless the determination of the test result and the gold standard are mutually blinded. Finally, in quantitative tests, like tumor markers, there is a risk of resubstitution bias if the sample used for defining the cutoff level is identical to the validation sample. Several practical recommendations are given for designing validation studies. In the second part, problems of the efficacy of follow-up programs in regard to patient survival are discussed. Five necessary conditions must be satisfied in order to make such an effect possible. Taking these aspects into account, it can be shown that the usual arguments advanced in favor of diagnostic monitoring are not stringent. As for the design of studies on the efficacy of cancer surveillance, non-randomized studies are inconclusive given that they suffer from a variety of specific biases. It can be shown that the demand, often found in recent publications, that comparative studies of more versus less intensive monitoring be designed so as to permit the detection of a survival advantage in the subgroup of patients who relapse, is unjustified. In the final sections, an overview over published randomized studies and meta-analyses on the efficacy of postoperative monitoring of breast and colorectal cancer is given.