Related Experiment Video
Updated: Jul 25, 2026
A Postoperative Evaluation Guideline for Computer-Assisted Reconstruction of the Mandible
Published on: January 28, 2020
Performance of an Algorithm Grading Surgery-Related Adverse Events According to the Clavien-Dindo Classification
Lisen Båverud Olsson1,2, Dennis Parkan1, Annika Sjövall1,2
1Department of Molecular Medicine and Surgery, Karolinska Institutet, Stockholm, Sweden.
Insights
An automated algorithm accurately grades surgery-related adverse events (AEs) using electronic health record (EHR) data. This tool shows strong agreement with manual grading, potentially improving patient care and reducing costs.
Area of Science:
- Medical Informatics
- Surgical Outcomes Research
- Health Data Science
Background:
- Surgery-related adverse events (AEs) are frequent, increasing patient morbidity and healthcare expenses.
- Manual chart review remains the standard for grading AEs, a resource-intensive process.
- Automated grading algorithms using electronic health record (EHR) data for Clavien-Dindo (C-D) classification have not been previously explored.
Purpose of the Study:
- To evaluate the performance of an automated algorithm for grading surgery-related adverse events (AEs) using the Clavien-Dindo (C-D) classification system.
- To compare the algorithm's grading accuracy against manual annotations by a colorectal surgeon (gold standard) and a research nurse (current practice).
Main Methods:
- The algorithm was developed using EHR data from Karolinska University Hospital Stockholm.
- It assigned a C-D grade to AEs within 30 days post-surgery for 1,379 elective colorectal procedures.
- Agreement was assessed using kappa statistics for C-D classification and intraclass correlation for the Comprehensive Complication Index (CCI) in a sample of 399 procedures.
Main Results:
- The algorithm demonstrated substantial agreement with surgeon (kappa=0.77) and nurse (kappa=0.74) for C-D classification.
- Agreement improved significantly after correcting misclassified surgeon annotations (kappa=0.89).
- Intraclass correlation for CCI was high between the algorithm and surgeon (0.89) after correction, and moderate with the nurse (0.76).
Conclusions:
- The developed algorithm shows promising performance for automated grading of surgery-related AEs.
- Implementation into real-time data with continuous evaluation is motivated to assess impact on AEs.
- Future work may involve using local EHR data to enhance risk prediction with machine learning techniques.
Objective:
To assess the performance of an algorithm for automated grading of surgery-related adverse events (AEs) according to Clavien-Dindo (C-D) classification.
Background:
Surgery-related AEs are common, lead to increased patient morbidity, and raise health care costs. Resource-intensive manual chart review is still standard, and, to our knowledge, algorithms using electronic health record (EHR) data to grade AEs according to C-D classification have not been explored.
Methods:
The algorithm was developed in a research database containing all EHR data of Karolinska University Hospital Stockholm and returns a C-D grade for each AE within 30 days. This raw score was used to grade the postoperative recovery of 1379 elective colorectal procedures according to C-D classification and Comprehensive Complication Index. Agreement with manual annotation of colorectal surgeon (gold standard) and research nurse (current practice) was assessed in a random sample of 399 procedures.
Results:
For the C-D classification, kappa was 0.77 (95% CI: 0.71 to 0.84) for algorithm versus surgeon and 0.74 (95% CI: 0.67 to 0.82) for algorithm versus nurse. The kappa value increased to 0.89 (95% CI: 0.84 to 0.95) after the correction of misclassified annotations by the surgeon. The intraclass correlation for Comprehensive Complication Index between algorithm and surgeon was 0.89 (95% CI: 0.87 to 0.91) after correction and 0.76 (95% CI: 0.71 to 0.80) for algorithm versus nurse.
Conclusions:
The performance of the algorithm motivates in our opinion implementation to real-time data under continuous scientific evaluation of the impact on AEs in different types of surgery. In the future, local EHR data could be used to enhance risk prediction with machine learning techniques.
