Predictive performance of Acute Physiological and Chronic Health Evaluation releases II to IV: a single New Zealand centre experience

There is debate in Australia and New Zealand around the appropriate use of illness severity scoring systems in Australasian intensive care units. The international benchmark is the Acute Physiological and Chronic Health Evaluation (APACHE) system. In order to compare the performance of recent APACHE releases, we audited 2080 sequential patients admitted between 1 January 2006 and 31 March 2008 to the Middlemore Hospital intensive care unit, Auckland, New Zealand. We compared the predictive performance of the proprietary APACHE II, IIIh, IIIj and IV releases, and the performance of a 'localised' version of APACHE II containing re-estimated coefficients derived from a legacy dataset (7703 sequential patients admitted between 1 January 1997 and 31 December 2005). Discrimination assessed by receiver operating characteristic curves was highest with the APACHE III and IV releases, and significantly better than the APACHE II releases. Calibration assessed by the Hosmer-Lemeshow statistic was poor with all releases, although it was best with APACHE IV and 'localised' version of the APACHE II release. Overall accuracy assessed by the Brier Mean Probability score and Shapiro's R statistic was best with APACHE IV. Our study suggests the possibility of improved prediction in moving to APACHE IV from older releases, although broader multicentre study within the Australian and New Zealand critical care community is warranted. Our study also suggests localisation of the APACHE system offers further opportunity to improve prediction, although these improvements may not be major without ground-up development of a new risk prediction model within our local critical care setting.