Severity measurement methods and judging hospital death rates for pneumonia

Payers and policymakers are increasingly examining hospital mortality rates as indicators of hospital quality. To be meaningful, these death rates must be adjusted for patient severity. This research examined whether judgments about an individual hospital's risk-adjusted mortality is affected by the severity adjustment method. Data came from 105 acute care hospitals nationwide that use the Medis-Groups severity measure. The study population was 18,016 adults hospitalized in 1991 for pneumonia. Multivariable logistic models to predict in-hospital death were computed separately for 14 severity methods, controlling for patient age, sex, and diagnosis-related group (DRG). For each hospital, observed-to-expected death rates and z scores were calculated for each severity method. The overall in-hospital death rate was 9.6%. Unadjusted mortality rates for the 105 hospitals ranged from 1.4% to 19.6%. After adjusting for age, sex, DRG, and severity, 73 facilities had observed mortality rates that did not differ significantly from expected rates according to all 14 severity methods; two had rates significantly higher than expected for all 14 severity methods. For 30 hospitals, observed mortality rates differed significantly from expected rates when judged by one or more but not all 14 severity methods. Kappa analysis showed fair to excellent agreement between severity methods. The 14 severity methods agreed about relative hospital performance more often than expected by chance, but perceptions of individual hospitals' mortality rates varied using different severity adjustment methods for almost one third of facilities. Judgments about individual hospital performance using different severity adjustment approaches may reach different conclusions.