The hazards of using administrative data to measure surgical quality

Administrative claims data have been used to measure risk-adjusted clinical outcomes of hospitalized patients. These data have been criticized because they cannot differentiate risk factors present at the time of admission from complications that occur during hospitalization. This paper illustrates how valid risk-adjustment can be achieved by enhancing administrative data with a present-on-admission code, admission laboratory data, and admission vital signs. Examples are presented for inpatient mortality rates following craniotomy and rates of postoperative sepsis after elective surgical procedures. Administrative claims data alone yielded a risk-adjustment model with 10 variables and a C-statistic of 0.891 for mortality after craniotomy, and a model with 18 variables and a C-statistic of 0.827 for postoperative sepsis. In contrast, the combination of administrative data and clinical data abstracted from medical records increased the number of variables in the craniotomy model to 21 with a C-statistic of 0.923, and the number of variables in the postoperative sepsis model to 29 with a C-statistic of 0.858. Use of only administrative data resulted in unacceptable amounts of systematic bias in 24 per cent of hospitals for craniotomy and 19 per cent of hospitals for postoperative sepsis. Addition of a present-on-admission code, laboratory data, and vital signs reduced the percentage of hospitals with unacceptable bias to two percent both for craniotomy and for postoperative sepsis. These illustrations demonstrate suboptimal risk stratification with administrative claims data only, but show that present-on-admission coding combined with readily available laboratory data and vital signs can support accurate risk-adjustment for the assessment of surgical outcomes.