Outcomes prediction in longitudinal data: Study designs evaluation, use case in ICU acquired sepsis

Area of Science:

• Critical Care Medicine
• Biomedical Informatics
• Data Science

Background:

• Outcome prediction in Electronic Health Records (EHR), particularly in critical care, is a growing research area.
• Existing evaluation approaches for longitudinal EHR data often fail to reflect real-world conditions.
• Focusing on ICU-acquired sepsis presents unique challenges due to sparse longitudinal data.

Purpose of the Study:

• To develop and evaluate improved methods for predicting ICU-acquired sepsis using longitudinal EHR data.
• To compare different temporal abstraction and pattern discovery techniques for feature engineering.
• To assess the suitability of various study designs for evaluating prediction models under real-life conditions.

Main Methods:

• Employed Temporal Abstraction and Time Interval-Related Patterns (TIRP) discovery for feature engineering from sparse longitudinal ICU data.
• Designed two experiments to predict ICU-acquired sepsis, one focusing on timing and the other on occurrence.
• Compared three epidemiological study designs, including case-crossover-control, for model evaluation under varying real-life conditions.

Main Results:

• Temporal Discretization for Classification (TD4C) outperformed Equal-Width Discretization, Knowledge-Based, and SAX.
• Two-state abstraction yielded better results than three or four states.
• XGBoost classifier demonstrated superior performance compared to Logistic Regression, Neural Net, and Random Forest.
• The default Binary TIRP representation method was more effective than Mean Duration and Horizontal Support variations.

Conclusions:

• TD4C and Binary TIRP representation combined with XGBoost offer a robust approach for ICU-acquired sepsis prediction.
• The case-crossover-control design is the most appropriate for evaluating prediction models in real-world critical care settings.
• Careful consideration of study design is crucial to avoid inflated performance metrics and ensure clinical applicability.