Comparing continuous versus categorical measures to assess and benchmark intensive care unit performance

Area of Science:

• Critical Care Medicine
• Health Services Research
• Performance Measurement

Background:

• Evaluating Intensive Care Unit (ICU) performance is crucial for quality improvement.
• Standardized Mortality Ratio (SMR) and Standardized Resource Use (SRU) are key performance indicators.
• Different methods exist for combining these metrics, impacting interpretation and utility.

Purpose of the Study:

• To compare categorical and continuous combinations of SMR and SRU for ICU performance evaluation.
• To assess the statistical properties and interpretability of different metric combinations.

Main Methods:

• Analysis of adult ICU admissions data from Brazil/Uruguay (2016-2018) and The Netherlands (2016-2018).
• Calculation of SMR and SRU using SAPS-3 (Brazil/Uruguay) and APACHE-IV (The Netherlands).
• Comparison of a categorical 'efficiency matrix' with a continuous 'average standardized ratio' (ASER).
• Evaluation of metric associations using Spearman's rho and R².

Main Results:

• Included 277,459 (Brazil/Uruguay) and 164,399 (Netherlands) ICU admissions.
• Median ASER was 0.99 in both regions, indicating overall expected performance.
• SMR and SRU showed higher correlation in Brazil/Uruguay ICUs (Spearman's Rho: 0.54) compared to Dutch ICUs (0.24).
• ASER values clustered in the least and most efficient groups, aligning with performance extremes.

Conclusions:

• Categorical metric combinations are easily interpretable but limit statistical inference for benchmarking.
• Continuous combinations, like ASER, provide superior statistical properties for performance evaluation, especially when metrics are positively correlated.
• The choice of metric combination impacts the ability to statistically benchmark ICU performance.