Assessment of inter-observer reliability of two five-level triage and acuity scales: a randomized controlled trial

The Emergency Severity Index (ESI) is an initial measure of patient assessment in the emergency department (ED). It rates patients based on acuity and predicted resource intensity from Level 1 (most ill) to Level 5 (least resource intensive). Already implemented and evaluated in several US hospitals, ESI has yet to be evaluated in a Canadian setting or compared with the five-level Canadian Emergency Department Triage and Acuity Scale (CTAS).

To compare the inter-observer reliability of 2 five-level triage and acuity scales.

Ten triage nurses, who had all been trained in the use of CTAS, from 4 urban, academic Canadian EDs were randomly assigned either to training in ESI version 3 (ESI v.3) or to refresher training in CTAS. They independently assigned triage scores to 200 emergency cases, unaware of the rating by the other nurses.

Number of years of nursing practice was the only significant demographic difference found between the 2 groups (p = 0.014). A quadratically weighted kappa to measure the inter-observer reliability of the CTAS group was 0.91 (0.90, 0.99) and not significantly different from that of the ESI group 0.89 (0.88, 0.99). An inter-test generalizability (G) study performed on the variance components derived from an analysis of variance (ANOVA) revealed G(5) = 0.90 (0.82, 0.99).

After 3 hours of training, experienced triage nurses were able to perform triage assessments using ESI v.3 with the same inter-observer reliability as those with experience and refresher training in using the CTAS.