An interpretable machine learning model for predicting emergence agitation in children: a multicenter development and
Qingyu Zhao1, Yi Zhang2, Rugang An3
1Department of Anesthesiology, The First Affiliated Hospital of Army Medical University (Southwest Hospital), Chongqing, 400038, China.
Insights
This study developed a machine learning model to predict emergence agitation (EA) in pediatric patients, identifying key risk factors like parental education and ALT levels. The model aids in early risk stratification for improved perioperative safety.
Area of Science:
- Pediatric Anesthesiology
- Machine Learning in Healthcare
- Perioperative Medicine
Background:
- Emergence agitation (EA) is a frequent complication in pediatric surgery, impacting patient safety.
- Early identification of EA risk factors is crucial for effective perioperative management.
Purpose of the Study:
- To identify risk factors for pediatric EA.
- To develop and validate an interpretable machine learning model for predicting EA risk.
Main Methods:
- A multicenter retrospective study of 445 pediatric patients.
- Feature selection using LASSO regression and evaluation of six machine learning algorithms (LR, SVM, MLP, RF, XGB, LGBM).
- Model interpretability assessed using SHapley Additive Explanations (SHAP).
Main Results:
- Five key predictors identified: parental education, ALT level, analgesic pump use, antagonist administration, and suctioning maneuvers.
- The Multilayer Perceptron (MLP) model showed optimal performance in external validation (AUC 0.705).
- SHAP analysis highlighted parental education, ALT, analgesic pump use, antagonist, and suctioning as significant risk factors.
Conclusions:
- An interpretable machine learning model was developed and externally validated for predicting pediatric EA risk.
- The model incorporates readily available clinical variables for risk stratification.
- Further prospective studies are needed to confirm clinical utility and generalizability.
Background:
Postoperative agitation (EA) is a common complication in pediatric patients, and its early identification is crucial for improving perioperative safety. This study aims to identify the risk factors for EA and develop an interpretable machine learning model.
Methods:
This multicenter retrospective study included 445 pediatric patients. Data from 321 patients from one center were used for model development, and 124 patients from another center were selected as an independent validation set. The development dataset was randomly divided into training and validation sets in a 8:2 ratio. Feature selection was performed using the Least Absolute Shrinkage and Selection Operator (LASSO) regression. Six machine learning algorithms were used to build the prediction model: Logistic Regression (LR), Support Vector Machine (SVM), Multilayer Perceptron (MLP), Random Forest (RF), Extreme Gradient Boosting (XGB), and Light Gradient Boosting Machine (LGBM). Model performance was evaluated based on the area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, specificity, Brier, and F1 score. The interpretability of the model was analyzed using the SHapley Additive Explanations (SHAP) method.
Result:
The incidence of EA in the development cohort and external validation cohort was 29.5% and 25.8%, respectively. Following feature selection, five clinically relevant predictive variables were identified: parental education level, ALT level, postoperative analgesic pump use, antagonist administration, and the number of suctioning maneuvers during extubation.In internal validation, the support vector machine (SVM) model achieved the best performance, with an AUC of 0.918 (95% confidence interval [CI] 0.844-0.973). In external validation, the MLP performed optimally, with an AUC of 0.705 (95% CI 0.590-0.804), accuracy of 0.718, sensitivity of 0.645, specificity of 0.780, F1 score of 0.571, and Brier score of 0.190.Given that external validation represents the gold standard for assessing model generalizability, MLP was chosen as the candidate model for clinical application. Using the optimal SVM model derived from internal validation, SHAP analysis demonstrated that shorter recovery time, analgesic pump use, higher parental education level, elevated ALT, no antagonist use, and absence of suctioning during extubation were significant risk factors for EA.Notably, recovery time-an intraoperative indicator-was excluded from the final clinical model to ensure that all predictive variables could be obtained prior to emergence from anesthesia.
Conclusion:
This study developed and externally validated an interpretable machine learning model for predicting the risk of emergence agitation (EA) in children undergoing elective surgery, incorporating five preoperative or postoperative readily available clinical variables: parental education level, ALT level, postoperative analgesic pump use, antagonist administration, and the number of suctioning maneuvers during extubation. The model exhibited moderate discriminatory performance, and SHAP analysis further clarified the contribution and underlying mechanisms of key risk factors. This model may serve as a preliminary decision-support tool for individualized risk stratification of pediatric EA. Nevertheless, future multicenter prospective studies are warranted to validate its generalizability and clinical utility prior to routine implementation.
