Related Experiment Video
Updated: May 1, 2026
Using Visual and Narrative Methods to Achieve Fair Process in Clinical Care
Published on: February 16, 2011
Benchmarking trauma centers on mortality alone does not reflect quality of care: implications for pay-for-performance
Zain G Hashmi1, Eric B Schneider, Renan Castillo
1From the Center for Surgical Trials and Outcomes Research (Z.G.H., E.B.S., E.R.H., A.H.H.), and Division of Acute Care Surgery, Trauma, Emergency Surgery and Critical Care (E.R.H., A.H.H.), Department of Surgery, Department of Emergency Medicine (ERH), and Department of Anesthesiology and Critical Care Medicine (A.L.), The Johns Hopkins School of Medicine; and Department of Health Policy and Management (R.C., A.H.H.), Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland; and Department of Surgery (S.N.Z., E.C.C.), Howard University College of Medicine, Washington, District of Columbia.
Background:
Trauma centers are currently benchmarked on mortality outcomes alone. However, pay-for-performance measures may financially penalize centers based on complications. Our objective was to determine whether the results would be similar to the current standard method of mortality-based benchmarking if trauma centers were profiled on complications.
Methods:
We analyzed data from the National Trauma Data Bank from 2007 to 2010. Patients 16 years or older with blunt or penetrating injuries and an Injury Severity Score (ISS) of 9 or higher were included. Risk-adjusted observed-to-expected (O/E) mortality ratios for each center were generated and used to rank each facility as high, average, or low performing. We similarly ranked facilities on O/E morbidity ratios defined as occurrence of any major complication. Concordance between hospital performance rankings was evaluated using a weighted κ statistic. Correlation between morbidity- and mortality-based O/E ratios was assessed using Pearson coefficients. Sensitivity analyses were performed to mitigate the competing risk of death for the morbidity analyses.
Results:
A total of 449,743 patients from 248 facilities were analyzed. The unadjusted morbidity and mortality rates were 10.0% and 6.9%, respectively. No correlation was found between morbidity- and mortality-based O/E ratios (r = -0.01). Only 40% of the centers had similar performance rankings for both mortality and morbidity. Of the 31 high performers for mortality, only 11 centers were also high performers for morbidity. A total of 78 centers were ranked as average, and 11 ranked as low performers on both outcomes. Comparison of hospital performance status using mortality and morbidity outcomes demonstrated poor concordance (weighted κ = 0.03, p = 0.22).
Conclusion:
Mortality-based external benchmarking does not identify centers with high complication rates. This creates a dichotomy between current trauma center profiling standards and measures used for pay-for-performance. A benchmarking mechanism that reflects all measures of quality is needed.
Level Of Evidence:
Prognostic/epidemiologic study, level III.
Related Concept Videos
01:15Methods of Documentation VI: Case Management Model
For example, a patient with a chronic...
01:25Errors occurring during blood pressure monitoring
Several factors...
01:27Ethical Issues
Ethical Concerns in Healthcare:
01:30Health Information Technology and Healthcare Information System
Health Information Technology, commonly called HIT, integrates advanced information systems and technology in healthcare settings. Its primary functions include:
01:22Standards of Care I
01:19Standards of Care II