Comparison of intensive care outcome prediction models based on admission scores with those based on 24-hour data

We compared the performance of six outcome prediction models--three based on 24-hour data and three based on admission-only data--in a metropolitan university-affiliated teaching hospital with a 10-bed intensive care unit. The Acute Physiology and Chronic Health Evaluation models, version II (APACHE II) and version III-J, and the Simplified Acute Physiology Score version II (SAPS II) are based on 24-hour data and were compared with the Mortality Prediction Model version II and the SAPS version III using international and Australian coefficients (SAPS IIIA). Data were collected prospectively according to the standard methodologies for each model. Calibration and discrimination for each model were assessed by the standardised mortality ratio, area under the receiver operating characteristic plot and Hosmer-Lemeshow contingency tables and chi-squared statistics (C10 and H10). Predetermined criteria were area under the receiver operating characteristic plot > 0.8, standardised mortality ratio 95% confidence interval includes 1.0, and C10 and H10 P values >0.05. Between October 1, 2005 and December 31, 2007, 1843 consecutive admissions were screened and after the standard exclusions, 1741 were included in the analysis. The SAPS II and SAPS IIIA models fulfilled and the APACHE II model failed all criteria. The other models satisfied the discrimination criterion but significantly over-predicted mortality risk and require recalibration. Outcome prediction models based on admission-only data compared favourably to those based on 24-hour data.