Related Experiment Video
Updated: Jun 10, 2025
Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
Published on: May 15, 2020
Development and validation of a novel tool for identification and categorization of non-technical errors associated
Jesse D Ey1, Victoria Kollias1, Matheesha B Herath1
1Department of Surgery, University of Adelaide, The Queen Elizabeth Hospital, Woodville, South Australia, Australia.
Background:
Up to half of all surgical adverse events are due to non-technical errors, making non-technical skill assessment and improvement a priority. No specific tools are available to retrospectively identify non-technical errors that have occurred in surgical patient care. This original study aimed to develop and provide evidence of validity and inter-rater reliability for the System for Identification and Categorization of Non-technical Error in Surgical Settings (SICNESS).
Methods:
A literature review, modified Delphi process, and two pilot phases were used to develop and test the SICNESS tool. For each pilot, 12 months of surgical mortality data from the Australian and New Zealand Audit of Surgical Mortality were assessed by two independent reviewers using the SICNESS tool. Main outcomes included tool validation through modified Delphi consensus, and inter-rater reliability for: non-technical error identification and non-technical error categorization using Cohen's κ coefficient, and overall agreement using Fleiss' κ coefficient.
Results:
Version 1 of the SICNESS was used for pilot 1, including 412 mortality cases, and identified and categorized non-technical errors with strong-moderate inter-rater reliability. Non-technical error exemplars were created and validated through Delphi consensus, and a novel mental model was developed. Pilot 2 included an additional 432 mortality cases. Inter-rater reliability was near perfect for leadership (κ 0.92, 95% c.i. 0.82 to 1.00); strong for non-technical error identification (κ 0.89, 0.84 to 0.93), communication and teamwork (κ 0.89, 0.79 to 0.99), and decision-making (κ 0.85, 0.79 to 0.92); and moderate for situational awareness (κ 0.79, 0.71 to 0.87) and overall agreement (κ 0.69, 0.66 to 0.73).
Conclusion:
The SICNESS is a reliable and valid tool, enabling retrospective identification and categorization of non-technical errors associated with death, occurring in real surgical patient interactions.
Related Concept Videos
01:10Documentation of Nursing Diagnosis
In some settings, data-driven computerized decision support systems are in place, allowing for more accurate nursing diagnoses. The database within one of these systems includes diagnostic labels defining characteristics, activities, and indicators for nursing. A nurse enters...
01:25Errors occurring during blood pressure monitoring
Several factors...