Beyond Random Splitting: Evaluating the Impact of Data Partitioning Strategies on Ventilator-Associated Pneumonia...

Area of Science:

• Critical Care Medicine
• Health Informatics
• Machine Learning in Healthcare

Background:

• Ventilator-Associated Pneumonia (VAP) is a critical care complication.
• Existing VAP prediction models often neglect the meronomic structure of healthcare data.
• Accurate VAP prediction is vital for improving patient outcomes in intensive care units.

Purpose of the Study:

• To develop and evaluate VAP prediction models that incorporate the meronomic structure of healthcare data.
• To compare the performance of different data splitting strategies for VAP prediction.
• To identify key clinical features predictive of VAP.

Main Methods:

• Utilized the MIMIC-III database, extracting data from clinical notes and structured fields.
• Identified 679 VAP and 3,207 non-VAP cases.
• Compared four data splitting strategies: Ventilator Session-Based Split and Hospital Admission-Based Split, with variations for single ICU stays.
• Evaluated four machine learning models.

Main Results:

• Conventional random splitting yielded moderate VAP prediction performance (AUROC: 76-81%).
• Restricting data splitting to single ICU stays significantly improved VAP prediction accuracy (AUROC: 86-87%).
• Hospital admission-based splits provided realistic performance (AUROC: 72-76%).
• Key predictors included mechanical ventilation hours, systolic blood pressure, and urine counts.

Conclusions:

• Robust VAP prediction necessitates evaluation frameworks that acknowledge healthcare data's meronomic nature.
• Data splitting strategies significantly influence VAP model performance.
• Single ICU stay-based analysis offers a promising approach for enhancing VAP prediction accuracy.