Control limits to identify outlying hospitals based on risk-stratification

Outcome indicators are routinely used to compare hospitals with respect to quality of care. Indicators might be based on observed proportions of adverse events (binary outcomes) or observed averages of e.g. lengths or costs of hospital stays (continuous outcomes). These observed values are compared with expected ones in an average hospital, which might be estimated from a reference sample and should be appropriately adjusted for the case mix of patients. One possibility to achieve a reliable adjustment is to stratify the patients according to their risks, where each patient belongs to one and only one stratum. Control limits calculated under the null hypothesis of an average hospital, allowing to decide whether a discrepancy between an observed and an expected value might be explained by chance or not, are then plotted around the indicator, such that hospitals falling above those control limits are detected as being statistically worse than an average hospital. Calculation of valid control limits is however not always obvious. In this article, we propose a simple and unified framework to calculate such control limits when adjustment is based on stratification, where we allow to distinguish and disentangle the variability explained by stratification and the variability due to chance, where we take into account the uncertainty about the estimation of the expected values, and where it is possible not only to detect those hospitals which are statistically worse, but also those which are statistically much worse than an average hospital. The method applies both to binary and continuous outcomes and is illustrated on Swiss hospital discharge data.