Using Medicare claims data to assess provider quality for CABG surgery: does it work well enough?

To assess the relative abilities of clinical and administrative data to predict mortality and to assess hospital quality of care for CABG surgery patients.

1991-1992 data from New York's Cardiac Surgery Reporting System (clinical data) and HCFA's MEDPAR (administrative data). STUDY DESIGN/SETTING/SAMPLE: This is an observational study that identifies significant risk factors for in-hospital mortality and that risk-adjusts hospital mortality rates using these variables. Setting was all 31 hospitals in New York State in which CABG surgery was performed in 1991-1992. A total of 13,577 patients undergoing isolated CABG surgery who could be matched in the two databases made up the sample.

Hospital risk-adjusted mortality rates, identification of "outlier" hospitals, and discrimination and calibration of statistical models were the main outcome measures.

Part of the discriminatory power of administrative statistical models resulted from the miscoding of postoperative complications as comorbidities. Removal of these complications led to deterioration in the model's C index (from C = .78 to C = .71 and C = .73). Also, provider performance assessments changed considerably when complications of care were distinguished from comorbidities. The addition of a couple of clinical data elements considerably improved the fit of administrative models. Further, a clinical model based on Medicare CABG patients yielded only three outliers, whereas eight were identified using a clinical model for all CABG patients.

If administrative databases are used in outcomes research, (1) efforts to distinguish complications of care from comorbidities should be undertaken, (2) much more accurate assessments may be obtained by appending a limited number of clinical data elements to administrative data before assessing outcomes, and (3) Medicare data may be misleading because they do not reflect outcomes for all patients.