Evaluation of the Effectiveness of a Surgical Checklist in Medicare Patients

Surgical checklists are increasingly used to improve compliance with evidence-based processes in the perioperative period. Although enthusiasm exists for using checklists to improve outcomes, recent studies have questioned their effectiveness in large populations.

We sought to examine the association of Keystone Surgery, a statewide implementation of an evidence-based checklist and Comprehensive Unit-based Safety Program, on surgical outcomes and health care costs.

We performed a study using national Medicare claims data for patients undergoing general and vascular surgery (n=1,002,241) from 2006 to 2011. A difference-in-differences approach was used to evaluate whether implementation was associated with improved surgical outcomes and decreased costs when compared with a national cohort of nonparticipating hospitals. Propensity score matching was used to select 10 control hospitals for each participating hospital. Costs were assessed using price-standardized 30-day Medicare payments for acute hospitalizations, readmissions, and high-cost outliers.

Keystone Surgery implementation in participating centers (N=95 hospitals) was not associated with improved outcomes. Difference-in-differences analysis accounting for trends in nonparticipating hospitals (N=950 hospitals) revealed no differences in adjusted rates of 30-day mortality [relative risk (RR)=1.03; 95% confidence intervals (CI), 0.97-1.10], any complication (RR=1.03; 95% CI, 0.99-1.07), reoperations (RR=0.89; 95% CI, 0.56-1.22), or readmissions (RR=1.01; 95% CI, 0.97-1.05). Medicare payments for the index admission increased following implementation ($516 average increase in payments; 95% CI, $210-$823 increase), as did readmission payments ($564 increase; 95% CI, $89-$1040 increase). High-outlier payments ($965 increase; 95% CI, $974decrease to $2904 increase) did not change.

Implementation of Keystone Surgery in Michigan was not associated with improved outcomes or decreased costs in Medicare patients.