Reliability adjustment: a necessity for trauma center ranking and benchmarking

Area of Science:

• Trauma Surgery
• Health Services Research
• Biostatistics

Background:

• Trauma center quality is benchmarked using observed-expected mortality ratios.
• Low patient volume can lead to unreliable mortality estimates in benchmarking.
• Reliability adjustment (RA) is a statistical method to mitigate bias from low volume.

Purpose of the Study:

• To explore the impact of reliability adjustment (RA) on trauma center benchmarking.
• To assess how RA affects risk-adjusted mortality rates and hospital rankings.
• To determine if RA improves the accuracy of quality assessments for trauma centers.

Main Methods:

• Analysis of the 2010 National Trauma Data Bank (NTDB) including 557 facilities and 278,558 patients.
• Risk-adjusted mortality rates were generated using hierarchical logistic regression.
• Reliability adjustment (RA) was applied using an empiric Bayes approach to modify rates.

Main Results:

• RA significantly reduced interfacility variation in mortality rates from 14-fold to 2-fold.
• The impact of RA was most pronounced for smaller, lower-volume trauma centers.
• RA reclassified 68 "best" and 18 "worst" hospitals, indicating significant shifts in performance assessment.

Conclusions:

• Reliability adjustment effectively reduces statistical noise in risk-adjusted mortality rates.
• RA is crucial for accurate benchmarking, particularly for low-volume trauma centers.
• Failure to use RA may lead to inappropriate placement of nearly one in six hospitals in performance rankings.