Related Experiment Video
Updated: May 11, 2026
Flexible Colonoscopy in Mice to Evaluate the Severity of Colitis and Colorectal Tumors Using a Validated Endoscopic Scoring System
Published on: October 16, 2013
Reliability of evaluating hospital quality by colorectal surgical site infection type
Elise H Lawson1, Clifford Y Ko, John L Adams
1*Department of Surgery, David Geffen School of Medicine, University of California, Los Angeles †Division of Research and Optimal Patient Care, American College of Surgeons, Chicago, IL ‡VA Greater Los Angeles Healthcare System, Los Angeles, California §RAND Corporation, Santa Monica, CA; and ¶Department of Surgery, Washington University School of Medicine in St Louis, Barnes Jewish Hospital, Saint Louis Veterans Affairs Medical Center, and Washington University Center for Health Policy and Olin Business School, St Louis, MO.
Objective:
To determine whether risk-adjusted colorectal SSI rates are statistically reliable as hospital quality measures.
Background:
Policymakers use surgical site infections (SSI) for public reporting of hospital quality and pay-for-performance because they are a relatively common and costly cause of patient morbidity.
Methods:
Patients who underwent a colorectal procedure in 2009 were identified from the American College of Surgeons National Surgical Quality Improvement Program. We developed hierarchical multivariate logistic models for (1) superficial SSI, (2) deep/organ-space SSI, and (3) "any SSI" and compared how each model ranked hospital-level risk-adjusted performance. Statistical reliability of hospital quality measurements was estimated on a scale from 0 to 1; with 0 indicating that apparent variation between a hospital's quality measurement and the average hospital is statistically unreliable, and 1 indicating that any observed variation is due to a real difference in performance.
Results:
Mean reliability of hospital-level quality measurements was 0.650 for superficial, 0.404 for deep/organ-space, and 0.586 for "any SSI." Lower reliability was accounted for by relatively little variation in risk-adjusted SSI rates between hospitals and insufficient numbers of colorectal cases submitted by individual hospitals. In 2009, we estimate that 22.1% of all US hospitals performed a sufficient number of colorectal cases to report superficial SSI rates at a high standard of statistical reliability and 1.0% did for deep/organ-space SSI.
Conclusions:
As currently constructed, colorectal SSI quality measures might not meet a high standard of statistical reliability for most hospitals, limiting their ability to confidently differentiate high and low performance. Despite an expectation of improving statistical power, combining superficial and deep/organ-space SSI into an "any SSI" measure worsens reliability.
