Strategies for validation

The easy-to-use statistical package has imposed a new hardship on the clinical researcher: too much complicated analysis. The problem is most acute in the interpretation of multivariate results that select a combination of several factors that "best" predict or explain medical outcomes. For example, these methods give rise to formulas that 1) weigh together the risk factors of smoking, blood pressure, and lipid levels as determinants of heart disease, or 2) construct from pathologic and clinical evidence a prognostic profile for disease-free rectal cancer patients. To help the clinician apply these methods, we propose that, on request, statistical packages also produce two sets of calculations that validate the primary analysis: 1) a set of simple tabulations that show how the factors and outcomes used in the primary analysis relate to one another, and 2) the results of alternative analyses that show factors which every analysis selects, factors which only appear in the primary analysis, and those which tend to substitute for one another.