A comparison of clinical registry versus administrative claims data for reporting of 30-day surgical complications

To compare the recording of 30-day postoperative complications between a national clinical registry and Medicare inpatient claims data and to determine whether the addition of outpatient claims data improves concordance with the clinical registry.

Policymakers are increasingly discussing use of postoperative complication rates for value-based purchasing. There is debate regarding the optimal data source for such measures.

Patient records (2005-2008) from the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) were linked to Medicare inpatient and outpatient claims data sets. We assessed the ability of (1) Medicare inpatient claims and (2) Medicare inpatient and outpatient claims to detect a core set of ACS-NSQIP 30-day postoperative complications: superficial surgical site infection (SSI), deep/organ-space SSI, any SSI (superficial and/or deep/organ-space), urinary tract infection, pneumonia, sepsis, deep venous thrombosis (DVT), pulmonary embolism, venous thromboembolism (DVT and/or pulmonary embolism), and myocardial infarction. Agreement of patient-level complications by ACS-NSQIP versus Medicare was assessed by κ statistics.

A total of 117,752 patients from more than 200 hospitals were studied. The sensitivity of inpatient claims data for detecting ACS-NSQIP complications ranged from 0.27 to 0.78; the percentage of false-positives ranged from 48% to 84%. Addition of outpatient claims data improved sensitivity slightly but also greatly increased the percentage of false-positives. Agreement was routinely poor between clinical and claims data for patient-level complications.

This analysis demonstrates important differences between ACS-NSQIP and Medicare claims data sets for measuring surgical complications. Poor accuracy potentially makes claims data suboptimal for evaluating surgical complications. These findings have meaningful implications for performance measures currently being considered.