Predicting Multidrug-Resistant Pneumonia: An Interpretable Machine Learning Model Validated in US and Chinese Patient
Yuejiao Lan1,2, Zheng Zhang1, Naijin Wei1
1Changchun University of Chinese Medicine, Changchun, People's Republic of China.
Objective:
Multidrug-resistant organisms (MDROs) complicate hospital-acquired and ventilator-associated pneumonia (HAP/VAP). We aimed to develop and validate a machine learning (ML) model to predict MDR risk in pneumonia patients and assess its utility for clinical decision support.
Patients And Methods:
We developed multiple ML models using data from the MIMIC-IV database (n=802). Feature selection was performed using LASSO regression and chi-square tests. Four models-Logistic Regression, Random Forest, XGBoost, and LightGBM-were trained, tuned, and calibrated. Model performance was evaluated using ROC curves and calibration plots. The final model was selected based on discrimination, calibration, and interpretability (assessed via SHAP). External validation on an independent cohort from a Chinese tertiary hospital (n=213) demonstrated the reproducibility and generalizability of its performance.
Results:
Among the evaluated ML models, Logistic Regression demonstrated the best overall performance. On the MIMIC-IV internal test set, it achieved an area under the curve (AUC) of 0.798 (95% CI: 0.718-0.872) with an accuracy of 0.807. External validation on an independent Chinese cohort confirmed the model's robust generalizability, achieving an AUC of 0.845. SHAP analysis identified key predictive features consistently across both cohorts, including the systemic immune-inflammation index (SII), albumin level, C-reactive protein-to-albumin ratio (CAR), number of antibiotic classes, white blood cell count (WBC), and absolute lymphocyte count (LYM abs). All of these features were significantly associated with MDR risk.
Conclusion:
Multiple ML models effectively predicted MDR infections in pneumonia patients, with Logistic Regression exhibiting particularly strong overall performance. Model reliability was enhanced through feature selection and probability calibration, while interpretability was improved by SHAP analysis. External validation confirmed the generalizability of our approach, supporting its potential application in clinical infection control. Future studies should focus on validation and the integration of more diverse clinical data sources.
Insights
Machine learning models can predict multidrug-resistant organism (MDRO) risk in pneumonia patients. Logistic Regression showed strong performance, aiding clinical decisions for infection control.
Area of Science:
- Medical Informatics
- Machine Learning in Healthcare
- Infectious Disease Epidemiology
Background:
- Multidrug-resistant organisms (MDROs) pose a significant challenge in treating hospital-acquired and ventilator-associated pneumonia (HAP/VAP).
- Accurate prediction of MDRO risk is crucial for effective clinical decision support and infection control strategies.
Purpose of the Study:
- To develop and validate a machine learning (ML) model for predicting the risk of multidrug-resistant infections in pneumonia patients.
- To assess the clinical utility of the developed ML model for decision support in HAP/VAP management.
Main Methods:
- Developed and compared multiple ML models (Logistic Regression, Random Forest, XGBoost, LightGBM) using data from the MIMIC-IV database.
- Employed LASSO regression and chi-square tests for feature selection, followed by model tuning, calibration, and interpretability analysis using SHAP.
- Validated the best-performing model on an independent cohort from a Chinese tertiary hospital to assess generalizability.
Main Results:
- Logistic Regression demonstrated superior performance, achieving an AUC of 0.798 internally and 0.845 externally.
- Key predictors identified by SHAP analysis included the systemic immune-inflammation index (SII), albumin, C-reactive protein-to-albumin ratio (CAR), antibiotic classes, WBC, and LYM abs.
- The model's reliability was enhanced through feature selection and probability calibration, with robust generalizability confirmed via external validation.
Conclusions:
- Machine learning models, particularly Logistic Regression, can effectively predict MDRO risk in pneumonia patients.
- The validated model offers potential for clinical decision support in infection control for HAP/VAP.
- Further validation and integration of diverse clinical data are recommended for future development.
Related Concept Videos
01:15Impact of Pharmacokinetic–Pharmacodynamic Models: Regulatory Decisions
01:14Atypical Pneumonia