APACHE II and SAPS II are poorly calibrated in a Hong Kong intensive care unit

This study seeks to determine if the Acute Physiology and Chronic Health Evaluation II (APACHE II) and the new Simplified Acute Physiology Score (SAPS II) model predictions are well calibrated in our adult Intensive Care Unit (ICU) patient population. 1064 successive ICU discharges were enrolled with 222 deaths at hospital discharge. APACHE II predicted 287.44 deaths, thus giving an APACHE II standardised mortality ratio (SMR) of 0.77 (95% confidence interval +/- 0.07). SAPS II predicted 269.59 deaths, giving a SAPS II SMR of 0.82 (95% confidence interval +/- 0.07). The Hosmer-Lemeshow goodness-of-fit C statistic was 43.96 for APACHE II and 49.06 for SAPS II (P < 0.001 for both, inferring significant departures from the null hypotheses of good calibration). The area under the Receiver Operating Characteristic plot for SAPS II risk of death was 0.87 (95% confidence interval +/- 0.028) while that for APACHE II risk of death was 0.88 (95% confidence interval +/- 0.026). Although the APACHE II and SAPS II models provide good discriminatory performance this study finds the APACHE II and SAPS II models to be poorly calibrated in that they over-predict mortality in our ICU population.