Risk adjustment for comparing hospital quality with surgery: how many variables are needed?

The American College of Surgeons' National Surgical Quality Improvement Program (ACS NSQIP) will soon be reporting procedure-specific outcomes, and hopes to reduce the burden of data collection by collecting fewer variables. We sought to determine whether these changes threaten the robustness of the risk adjustment of hospital quality comparisons.

We used prospective, clinical data from the ACS NSQIP from 2005 to 2007 (184 hospitals, 74,887 patients). For the 5 general surgery operations in the procedure-specific NSQIP, we compared the ability of the full model (21 variables), an intermediate model (12 variables), and a limited model (5 variables) to predict patient outcomes and to risk-adjust hospital outcomes.

The intermediate and limited models were comparable with the full model in all analyses. In the assessment of patient risk, the limited and full models had very similar discrimination at the patient level (C-indices for all 5 procedures combined of 0.93 versus 0.91 for mortality and 0.78 versus 0.76 for morbidity) and showed good calibration across strata of patient risk. In assessing hospital-specific outcomes, results from the limited and full-risk models were highly correlated for both mortality (range 0.94 to 0.99 across the 5 operations) and morbidity (range 0.96 to 0.99).

Procedure-specific hospital quality measures can be adequately risk-adjusted with a limited number of variables. In the context of the ACS NSQIP, moving to a more limited model will dramatically reduce the burden of data collection for participating hospitals.