Development of a machine learning model and a web application for predicting neurological outcome at hospital
Kyota Kitagawa1, Satoshi Maki2, Takeo Furuya1
1Department of Orthopaedic Surgery, Graduate School of Medicine, Chiba University, Chiba, Japan.
Background:
Spinal cord injury (SCI) is a devastating condition with profound physical, psychological, and socioeconomic consequences. Despite advances in SCI treatment, accurately predicting functional recovery remains a significant challenge. Conventional prognostic methods often fall short in capturing the complex interplay of factors influencing SCI outcomes. There is an urgent demand for more precise and comprehensive prognostic tools that can guide clinical decision-making and improve patient care in SCI.
Purpose:
This study aims to develop and validate a machine learning (ML) model for predicting American Spinal Injury Association (ASIA) Impairment Scale (AIS) at discharge in SCI patients. We also aim to convert this model into an open-access web application.
Study Design/Setting:
This was a retrospective cohort study enrolling traumatic SCI patients from 1991 to 2015, analyzed in 2023. Data were obtained from the Japan Rehabilitation Database (JARD), a comprehensive nationwide database that includes SCI patients from specialized SCI centers and rehabilitation hospitals across Japan.
Patients Sample:
A total of 4,108 SCI cases from JARD were reviewed, excluding 405 cases, patients caused by nontraumatic injuries, patients who were graded as AIS E at admission, and patients without data of AIS at discharge, resulting in 3,703 cases being included in the study. Patient demographics and specific SCI injury characteristics at admission were utilized for model training and prediction.
Outcome Measures:
Model performance was evaluated based on R2, accuracy, and the weighted Kappa coefficient. Shapley additive explanations (SHAP) values highlighted significant features influencing the model's output.
Methods:
The primary outcome was AIS at discharge, treated as a continuous variable (0-4) to capture the ordinal nature and clinical significance of potential misclassifications. Data preprocessing included multicollinearity removal, feature selection using the Boruta algorithm, and iterative imputation for missing data. The dataset was split using the hold-out method with a 7:3 ratio resulting in 2,592 cases for training and 1,111 cases for testing the regression model. A best performing model was defined as the highest R2 using PyCaret's automated model comparison. Final predictions of regression model were discretized to the original AIS categories for clinical interpretation.
Results:
The Gradient Boosting Regressor (GBR) was identified as the optimal model. The GBR model showed an R² of 0.869, accuracy of 0.814, and weighted Kappa of 0.940. Eleven key variables, including AIS at admission, the day from injury to admission, and the motor score of L3, were identified as significant based on SHAP values. This model was then adapted into a web application via Streamlit.
Conclusions:
We developed a high-accuracy ML model for predicting the AIS at discharge, which effectively captures the ordinal nature of the AIS scale, using 11 key variables. This model demonstrated its performance to provide reliable prognostic information. The model has been integrated into a user-friendly, open-access web application (http://3.138.174.54:8502/). This tool has the potential to aid in resource allocation and enhance treatment for each patient.
