Accuracy and efficiency of an automated system for calculating APACHE II scores in an intensive care unit

We evaluated the reliability and efficiency of an automated system for calculating APACHE II scores. We imported an automated APACHE II scoring system developed at another institution. We scored a convenience sample of 50 consecutive intensive care unit (ICU) admissions using three methods: (1) the automated system (2) an expert scorer using a manual data abstraction method, and (3) the current manual scoring method used in the ICU. We analyzed interrater reliability among the three groups, and compared scoring time between the automated and the expert groups. Interrater reliability testing demonstrated a very high agreement between the computer and the expert scorers, (r = 0.97 p < 0.01) and a high agreement between the computer and the nursing staff (r = 0.80, p < 0.01). The mean time required to complete the data sheet manually and input data into a standalone computer manually was 4 minutes and 55 seconds, compared to total mean time of 33 seconds for the automated system. Automated APACHE II scoring yielded results more reliable than those of an expert scorer, as judged by a second researcher. The time required to score patients was reduced and reliability of scores was improved with use of an automated APACHE II scoring system.