Machine Learning Methods to Identify Pediatric Shunt Malfunction in the Acute Care Setting and the Development of...

Identifying predictors of shunt failure in acute care settings is challenging. Previous studies largely rely on traditional statistical analyses to evaluate risk factors associated with shunt failure. However, these methods have not been adapted for emergency department (ED) presentations, where machine learning (ML) techniques may offer enhanced predictive power. Thus, we explored the application of ML and artificial intelligence in predicting necessity for shunt revision in pediatric ED visits.

A retrospective analysis was conducted on 1167 pediatric neurosurgical consults for suspected shunt malfunction from 2017 to 2022. We collected 24 historical, clinical, and radiographic variables. Various ML techniques were used, including support vector machine, K-nearest neighbors, random forest, an artificial neural network (ANN), and a proprietary large language model named ShuntGPT (SGPT), which classifies the need for shunt revision.

Among 1167 ED consults for shunt malfunction, 285 patients (24.4%) underwent revision. Multivariate analysis identified increased ventricle size, abnormal shunt series, lethargy, altered mental status, and bradycardia as predictors of need for revision. The best-performing ML models included ANN (accuracy 84%, area under the curve [AUC] 0.88, 71% sensitivity, 88% specificity) and SGPT (accuracy 87%, precision 0.80, recall 0.64, AUC 0.927). Traditional classifiers (support vector machine, K-nearest neighbors, random forest) achieved AUCs of 0.81 to 0.86 with varying trade-offs in sensitivity and precision. With imaging results removed from training data, the performance of all models suffered, however, SGPT retained a high level of discrimination (AUC 0.84). SGPT continued to show high accuracy (83%) on a separate validation cohort.

Advanced ML models generally outperformed traditional statistical analyses, albeit with concerns about overfitting due to the extensive variable set. SGPT, in particular, showed superior performance, likely due to its capacity to interpret nuanced text. This model represents a promising tool to enhance decision-making in pediatric acute care settings regarding shunt malfunctions.