Assessment of Multimodal Natural Language Processing in Ascertaining Perioperative Safety Indicators From...

Standard spine surgery machine learning (ML) models often rely on structured clinical data, overlooking nuanced free text, such as preoperative surgical notes. The aims of this work were to develop a multimodal ML model combining structured electronic health record (EHR) data with natural language-processed unstructured clinical narratives.

After testing against Convolutional Neural Network, Support Vector Machine, LightGBM, and Random Forest algorithms, the XGBoost algorithm was selected for model development. Three models were developed: (1) a structured EHR-based ML model; (2) an NLP-based model using preoperative notes; (3) a combined multimodal model. Perioperative outcomes included extended length of stay (≥8.0 days) and nonhome discharge. Preprocessing included tokenization, stemming, and bag-of-words vectorization. Hyperparameters were tuned through grid search and 10-fold cross-validation. Key performance metrics included area-under-the-receiver-operating characteristic curve, Brier score, calibration slope and intercept, precision, recall, and F1 score.

A total of 486 patients (58.8% female, n = 281) were included, with a median age of 61.0 years (interquartile range: 52.0 to 68.0 years) and median body mass index of 29.4 kg/m 2 (interquartile range: 25.1 to 34.5 kg/m 2 ). For extended length of stay, the multimodal model excelled (ROC-AUC: 0.908, Brier: 0.114, F1: 0.896), followed by the NLP-only model (ROC-AUC: 0.868, Brier: 0.132, F1: 0.877), and the XGBoost-only model (ROC-AUC: 0.736, Brier: 0.201, F1: 0.815). For nonhome discharge, the multimodal model led (ROC-AUC: 0.920, Brier: 0.105, F1: 0.907), compared with the NLP-only model (ROC-AUC: 0.892, Brier: 0.102, F1: 0.916) and XGBoost-only model (ROC-AUC: 0.771, Brier: 0.144, F1: 0.893). Explainable AI revealed that body mass index, age, Medicare insurance, Charlson comorbidity index, Medicaid status, Hispanic ethnicity, fusion history, and thoracolumbar and cervical levels of surgery were the most important model features.

Incorporating unstructured surgeon notes into ML models markedly enhanced the prediction of perioperative outcomes in spinal surgery, suggesting that free-text notes may provide greater predictive utility than standard EHR variables.

III.