Ranking hospitals on surgical quality: does risk-adjustment always matter?

It is a widely held belief that detailed risk-adjustment is always necessary in comparative reports of surgical performance. We sought to evaluate the importance of risk-adjustment for two cardiac surgery report cards in New York and Pennsylvania.

We abstracted data directly from publicly available cardiac surgery report cards from New York State (2001 and 2002) and Pennsylvania (2000 and 2002). We first estimated the correlation between unadjusted and risk-adjusted mortality rates. We then divided hospitals into three groups of historic performance (best, average, and worst) for both unadjusted and risk-adjusted mortality rankings. We then calculated the risk-adjusted mortality within each of these groups using data from the report card from the subsequent year.

Risk-adjusted and unadjusted mortality rates were highly correlated for both New York (Pearson's r=0.95; Spearman's r=0.91) and Pennsylvania (Pearson's r=0.87; Spearman's r=0.89). For both states, risk-adjusted and unadjusted rankings were equally good at predicting subsequent mortality. In New York State, mortality for hospitals in the worst group was 50% higher than that in the best group regardless of whether unadjusted (relative risk [RR], 1.51) or adjusted (RR, 1.49) rankings were used. The same was found in Pennsylvania, where the results for unadjusted (RR, 1.53) and adjusted (RR, 1.45) rankings were nearly identical.

Based on data from two prominent state registries, risk-adjusted and unadjusted mortality rates provide nearly identical estimates of hospital performance with coronary artery bypass. Risk-adjustment may not always be important for identifying high quality hospitals.