Performance of an Algorithm Grading Surgery-Related Adverse Events According to the Clavien-Dindo Classification

To assess the performance of an algorithm for automated grading of surgery-related adverse events (AEs) according to Clavien-Dindo (C-D) classification.

Surgery-related AEs are common, lead to increased patient morbidity, and raise health care costs. Resource-intensive manual chart review is still standard, and, to our knowledge, algorithms using electronic health record (EHR) data to grade AEs according to C-D classification have not been explored.

The algorithm was developed in a research database containing all EHR data of Karolinska University Hospital Stockholm and returns a C-D grade for each AE within 30 days. This raw score was used to grade the postoperative recovery of 1379 elective colorectal procedures according to C-D classification and Comprehensive Complication Index. Agreement with manual annotation of colorectal surgeon (gold standard) and research nurse (current practice) was assessed in a random sample of 399 procedures.

For the C-D classification, kappa was 0.77 (95% CI: 0.71 to 0.84) for algorithm versus surgeon and 0.74 (95% CI: 0.67 to 0.82) for algorithm versus nurse. The kappa value increased to 0.89 (95% CI: 0.84 to 0.95) after the correction of misclassified annotations by the surgeon. The intraclass correlation for Comprehensive Complication Index between algorithm and surgeon was 0.89 (95% CI: 0.87 to 0.91) after correction and 0.76 (95% CI: 0.71 to 0.80) for algorithm versus nurse.

The performance of the algorithm motivates in our opinion implementation to real-time data under continuous scientific evaluation of the impact on AEs in different types of surgery. In the future, local EHR data could be used to enhance risk prediction with machine learning techniques.