Validation of the ACS-NSQIP Risk Calculator: A Machine-Learning Risk Tool for Predicting Complications and Mortality
Katherine E Pierce1, Bhaveen H Kapadia2, Sara Naessig1
1Departments of Orthopaedic and Neurologic Surgery, NYU Langone Orthopedic Hospital; New York Spine Institute, New York, New York, USA.
Objective:
To calculate the risk for postoperative complications and mortality after corrective surgery of adult spinal deformity (ASD) patients using the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) surgical risk calculator (SRC).
Methods:
Patients aged ≥18 years undergoing corrective surgery for ASD were identified. Current procedural terminology (CPT) codes of 22800, 22802, 22804, 22808, 22801, 22812, 22818, 22819, 22843, 22844, 22846, 22847, 22842, and 22845 were assessed if the patient had an International Classification of Diseases Ninth Revision (ICD-9) scoliosis diagnosis (737.00-737.9). Calculated perioperative complication risk averages via the ACS-NSQIP surgical calculator were compared with observed complication rates. Outcomes assessed were as follows: serious complication, any complication, pneumonia, cardiac complication, surgical site infection, urinary tract infection, venous thromboembolism, renal failure, readmission, return to operating room, death, discharge to nursing or rehabilitation, sepsis, and total length of hospital stay. Predictive performance of the calculator was analyzed by computation of the Brier score. A Brier score is the sum of squared differences between the binary outcome and the predicted risk and ranges from 0 to a maximum Brier score = (mean observed outcome)*(1-[mean observed outcome]). Values closer to 0 are suggestive of better predictive performance. Length of stay (LOS) was assessed with a Bland-Altman plot, which plots the average of observed LOS on the x axis and the difference between the observed and predicted LOS on the y axis.
Results:
A total of 9143 ASD patients (58.9 years, 56% females, 29.2 kg/m2) were identified; 36.9% of procedures involved decompression and 100% involved fusion. The means for individual patient characteristics entered into the online risk calculator interface were as follows: functional status (independent: 94.9%, partially dependent: 4.4%, totally dependent: 0.70%), 1.6% emergent cases, wound class (clean: 94.7%, clean/contaminated: 0.8%, contaminated: 0.5%, dirty/infected: 1.4%), American Society of Anesthesiologists class (I: 2.7%, II: 40.7%, III: 52.1%, IV: 4.6%, V: 0%), 5.1% steroid use for chronic condition, 0.04% ascites within 30 days prior to surgery, 1.73% systemic sepsis within 48 hours of surgery, 0.40% ventilator dependent, 3.2% disseminated cancer, 15.6% diabetes mellitus, 52.8% use of hypertensive medications, 0.3% congestive heart failure , 3% dyspnea, 21.4% history of smoking within 1 year, 4.3% chronic obstructive pulmonary disease, 0.7% dialysis, and 0.1% acute renal failure. Predictive of any 30-day postoperative complications ranged from 2.8 to 18.5% across CPT codes, where the actual rate in the cohort was 11.4%, and demonstrated good predictive performance via Brier score (0.000002, maximum: 0.101). The predicted and observed percentages for each of the 13 outcomes were assessed and their associated Brier scores and Brier maximums were calculated. Mean difference between observed and predicted LOS was 2.375 days (95% CI 9.895-5.145).
Conclusions:
The ACS-NSQIP SRC predicts surgical risk in patients undergoing ASD corrective surgery. This tool can be used as a resource in preoperative optimization by deformity surgeons.
Level Of Evidence:
3.
