An interpretable machine learning model for predicting emergence agitation in children: a multicenter development and
Qingyu Zhao1, Yi Zhang2, Rugang An3
1Department of Anesthesiology, The First Affiliated Hospital of Army Medical University (Southwest Hospital), Chongqing, 400038, China.
Background:
Postoperative agitation (EA) is a common complication in pediatric patients, and its early identification is crucial for improving perioperative safety. This study aims to identify the risk factors for EA and develop an interpretable machine learning model.
Methods:
This multicenter retrospective study included 445 pediatric patients. Data from 321 patients from one center were used for model development, and 124 patients from another center were selected as an independent validation set. The development dataset was randomly divided into training and validation sets in a 8:2 ratio. Feature selection was performed using the Least Absolute Shrinkage and Selection Operator (LASSO) regression. Six machine learning algorithms were used to build the prediction model: Logistic Regression (LR), Support Vector Machine (SVM), Multilayer Perceptron (MLP), Random Forest (RF), Extreme Gradient Boosting (XGB), and Light Gradient Boosting Machine (LGBM). Model performance was evaluated based on the area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, specificity, Brier, and F1 score. The interpretability of the model was analyzed using the SHapley Additive Explanations (SHAP) method.
Result:
The incidence of EA in the development cohort and external validation cohort was 29.5% and 25.8%, respectively. Following feature selection, five clinically relevant predictive variables were identified: parental education level, ALT level, postoperative analgesic pump use, antagonist administration, and the number of suctioning maneuvers during extubation.In internal validation, the support vector machine (SVM) model achieved the best performance, with an AUC of 0.918 (95% confidence interval [CI] 0.844-0.973). In external validation, the MLP performed optimally, with an AUC of 0.705 (95% CI 0.590-0.804), accuracy of 0.718, sensitivity of 0.645, specificity of 0.780, F1 score of 0.571, and Brier score of 0.190.Given that external validation represents the gold standard for assessing model generalizability, MLP was chosen as the candidate model for clinical application. Using the optimal SVM model derived from internal validation, SHAP analysis demonstrated that shorter recovery time, analgesic pump use, higher parental education level, elevated ALT, no antagonist use, and absence of suctioning during extubation were significant risk factors for EA.Notably, recovery time-an intraoperative indicator-was excluded from the final clinical model to ensure that all predictive variables could be obtained prior to emergence from anesthesia.
Conclusion:
This study developed and externally validated an interpretable machine learning model for predicting the risk of emergence agitation (EA) in children undergoing elective surgery, incorporating five preoperative or postoperative readily available clinical variables: parental education level, ALT level, postoperative analgesic pump use, antagonist administration, and the number of suctioning maneuvers during extubation. The model exhibited moderate discriminatory performance, and SHAP analysis further clarified the contribution and underlying mechanisms of key risk factors. This model may serve as a preliminary decision-support tool for individualized risk stratification of pediatric EA. Nevertheless, future multicenter prospective studies are warranted to validate its generalizability and clinical utility prior to routine implementation.
