Health-care quality registers: outcome-orientated ranking of hospitals is unreliable

Public disclosure of outcome-orientated ranking of hospitals is becoming increasingly popular and is routinely used by Swedish health-care authorities. Whereas uncertainty about an outcome is usually presented with 95% confidence intervals, ranking's based on the same outcome are typically presented without any concern for bias or statistical precision. In order to study the effect of incomplete registration of re-operation on hospital ranking we performed a simulation study using published data on the two-year risk of re-operation after total hip replacement. This showed that whereas minor registration incompleteness has little effect on the observed risk of revision, it can lead to major errors in the ranking of hospitals. We doubt whether a level of data entry sufficient to generate a correct ranking can be achieved, and recommend that when ranking hospitals, the uncertainties about data quality and random events should be clearly described as an integral part of the results.