Emergency department triage: do experienced nurses agree on triage scores?

The reproducibility of the Canadian Triage & Acuity Scale (CTAS), designed and introduced in the late 1990s in all Canadian emergency departments (EDs), has been studied mostly using measures of interrater agreement. However, each of these studies shares a common limitation: the nurses had received fresh CTAS training, which is likely to have led to an overestimation of the reproducibility of CTAS.

This study aims to assess the interrater reliability of the CTAS in current clinical practice, that is, as used by experienced ED nurses without recent certification or recertification.

A prospective sample of 100 patients arriving by ambulance was identified and yielded a set of 100 written scenarios. Five experienced ED nurses reviewed and blindly assigned a CTAS score to each scenario. The agreement among nurses was measured using the Kappa statistic calculated with quadratic weights. Kappa values were generated for each pair of nurses and a global Kappa coefficient was calculated to measure overall agreement.

Overall interrater agreement was moderate, with a global Kappa of 0.44 (95% confidence interval 0.40-0.48). However, pairwise, Kappa values were heterogeneous (0.30 to 0.61, p=0.0013).

The moderate interrater agreement observed in this study is disappointingly low and suggests that CTAS reliability may be lower than expected, and this warrants further research. Intra-observer reliability of CTAS should be ascertained more extensively among experienced nurses, and a future evaluation should involve several institutions.