Interrater Reliability in the Triage of Children With Mental and Behavioral Health Symptoms Using Two Triage Systems

The Australian mental health triage scale is widely used in Australia to triage emergency department patients with behavioral health symptoms. The Australian mental health triage scale outperforms the Emergency Severity Index in distinguishing levels of urgency and meeting time-to-evaluation thresholds among adult patients, but its performance among children is not well studied. We aimed to determine the interrater reliability of acuity levels assigned using the Australian mental health triage scale and the Emergency Severity Index for children with behavioral health emergency department presentations.

A convenience sample of triage-trained emergency nurses at 2 academic United States children's hospitals was surveyed. Nurses completed a brief training module describing behavioral health triage using the Australian mental health triage scale and the Emergency Severity Index. For 30 written case scenarios of emergency department presentations by children for behavioral health symptoms, nurses assigned acuity levels using the Australian mental health triage scale and the Emergency Severity Index. An interrater reliability using intraclass correlation coefficients was calculated.

Thirty participating emergency nurses reported a median of 7 years of experience working in the emergency department. Half of the acuity levels assigned using the Emergency Severity Index were level 2, whereas acuity levels assigned using the Australian mental health triage scale were more evenly distributed across levels. The intraclass correlation coefficient was 0.81 (95% CI, 0.72-0.89) for the Australian mental health triage scale and 0.63 (95% CI, 0.51-0.78) for the Emergency Severity Index.

Among triage-trained nurses using written case scenarios, interrater reliability was high for the Australian mental health triage scale and moderate for the Emergency Severity Index, which may inform consistency of care provision. The wider distribution of acuity levels generated by the Australian mental health triage scale may enable greater differentiation of illness severity. Prospective studies are needed to assess the reliability and validity of the Australian mental health triage scale among pediatric patients in the emergency department.