Assessment of predictive models for binary outcomes: an empirical approach using operative death from cardiac surgery

Predictive models in medical research have gained popularity among physicians as an important tool in medical decision making. Eight methodological strategies for creating predictive models are compared in a large, complex data base consisting of preoperative risk and operative outcome data on 12,712 patients undergoing coronary artery bypass grafting and entered into the Department of Veterans Affairs Cardiac Surgery Risk Assessment Program between April 1987 and March 1990. The models under consideration were developed to predict operative death (any death within 30 days following the surgical procedure or later if the result of a perioperative complication). The two strategies with the best predictive power among the eight examined were stepwise logistic regression alone and data reduction by cluster analysis combined with clinical judgement followed by a logistic regression model. The additive model based on unadjusted relative risks, the model based on Bayes' Theorem, and the logistic model using all candidate variables were good alternatives. Whether or not we imputed values did not have a significant impact on the predictive power of the models.