Performance of a Machine Learning Algorithm Using Electronic Health Record Data to Predict Postoperative...

Area of Science:

• Medical Informatics
• Surgical Outcomes Research
• Artificial Intelligence in Healthcare

Background:

• Predicting postoperative complications is crucial for shared decision-making, risk reduction, and resource allocation.
• Integrating accurate, real-time predictions into clinical workflows is essential but lacks high-level prospective evidence.
• AI predictive analytic platforms using automated electronic health record (EHR) data offer a potential solution.

Purpose of the Study:

• To evaluate the predictive performance of the MySurgeryRisk AI system during prospective validation.
• To assess the feasibility of delivering automated AI predictions directly to surgeons' mobile devices.

Main Methods:

• A prognostic study utilizing automated EHR data and machine learning (generalized additive and random forest models).
• Development using retrospective data (52,117 procedures) and validation on prospective data (22,300 procedures) from adult inpatient surgeries (June 2014-September 2020).
• Model performance assessed using Area Under the Receiver Operating Characteristic Curve (AUROC).

Main Results:

• The AI system demonstrated stable predictive performance, with AUROC values for various complications (e.g., sepsis 0.86, 30-day mortality 0.84) comparable to surgeon accuracy.
• A random forest model with 135 input features achieved high AUROC values across multiple complication types.
• The mobile application facilitated faster data access and displayed patient risk, key factors, and comparative complication patterns.

Conclusions:

• Automated, real-time AI predictions of postoperative complications delivered via mobile devices show strong performance in prospective clinical validation.
• The system's accuracy matches that of surgeons, indicating its potential utility in clinical practice.
• This technology supports informed surgical decision-making and risk management through accessible, data-driven insights.