Related Experiment Video
Updated: Jul 26, 2026
Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
Published on: May 15, 2020
Interrater Reliability, Accuracy, and Triage Time Pre- and Post-implementation of a Real-Time Electronic Triage
Shelley L McLeod1, Joy McCarron2, Tamer Ahmed2
1Schwartz/Reisman Emergency Medicine Institute, Sinai Health System, Toronto, Ontario, Canada; Department of Family and Community Medicine, Division of Emergency Medicine, University of Toronto, Toronto, Ontario, Canada; Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada.
The electronic Canadian Triage and Acuity Scale (eCTAS) improved triage score accuracy and consistency. This standardized electronic tool enhanced patient safety without significantly increasing assessment time.
Area of Science:
- Emergency Medicine
- Health Informatics
- Patient Safety
Background:
- The Canadian Triage and Acuity Scale (CTAS) is crucial for emergency department (ED) patient prioritization.
- Standardizing CTAS application is essential for improving patient safety and care quality.
- The electronic CTAS (eCTAS) was developed as a decision-support tool to standardize triage.
Purpose of the Study:
- To evaluate the interrater agreement of triage scores before and after implementing the eCTAS.
- To assess the impact of eCTAS on the accuracy of triage assessments.
- To determine if eCTAS affects the time required for triage.
Main Methods:
- A prospective, observational study was conducted across 7 diverse emergency departments.
- A provincial auditor independently assigned CTAS scores in real-time, comparing them with nurse-assigned scores.
- Interrater agreement was analyzed using kappa statistics (κ) pre- and post-eCTAS implementation.
Main Results:
- Exact agreement increased from 75.4% pre-eCTAS to 92.7% post-eCTAS.
- eCTAS significantly reduced both over-triaged (12.0% to 5.1%) and under-triaged (12.6% to 2.2%) patients.
- Interrater agreement improved substantially with eCTAS (κ 0.89 vs. 0.63), with a minimal increase in median triage time (312s vs. 347s).
Conclusions:
- Implementing eCTAS enhances the standardization and accuracy of emergency department triage.
- The electronic tool significantly improves interrater agreement for CTAS scores.
- eCTAS provides these benefits without a substantial increase in the time taken for triage assessments.
