Ranking hospitals: do we gain reliability by using composite rather than individual indicators?

Area of Science:

• Health Services Research
• Quality Improvement
• Healthcare Analytics

Background:

• Hospital quality indicators are widely used but their reliability in distinguishing true performance differences is unclear.
• Rankability quantifies the proportion of performance variation attributable to true hospital differences versus random variation.

Purpose of the Study:

• To assess if combining data into composite measures or using multi-year data enhances the reliability of hospital care quality indicators.
• To improve the detection of genuine hospital performance variations.

Main Methods:

• Utilized the Dutch National Medical Registration (2007-2012) for stroke, colorectal carcinoma, heart failure, acute myocardial infarction, and hip/knee arthroplasty.
• Calculated rankability for in-hospital mortality, 30-day readmission, and prolonged length of stay (LOS) using single years, 3-year periods, and dichotomous/ordinal composite measures.
• Rankability was defined as (between-hospital variation/total variation)×100% and classified as low (<50%), moderate (50%-75%), or high (>75%).

Main Results:

• Mortality rankability was generally low to moderate; acute readmission rankability was low, with exceptions for acute myocardial infarction.
• Prolonged LOS rankability was at least moderate. Combining multiple years improved rankability but remained low in some cases.
• Dichotomous composite measures showed at least moderate rankability across all diagnoses (51%-96%). Ordinal composites also demonstrated high rankability, except for heart failure (46%).

Conclusions:

• Combining data from multiple years or creating composite indicators significantly improves the reliability of hospital rankings.
• Composite measures offer more comprehensive information and reliable rankings compared to aggregating data from multiple years.
• These enhanced methods improve the ability to identify true hospital performance differences, particularly for mortality and acute readmission indicators.