External validation of a machine learning model to predict hemodynamic instability in intensive care unit

Area of Science:

• Critical Care Medicine
• Machine Learning in Healthcare
• Predictive Analytics

Background:

• Early prediction of hemodynamic instability is crucial for improving critical care outcomes.
• External validation of predictive models like the Hemodynamic Stability Index (HSI) is limited, particularly for generalizability in diverse populations.
• This study focuses on validating the HSI model in Asian patients.

Purpose of the Study:

• To independently validate the Hemodynamic Stability Index (HSI), a multi-parameter machine learning model, for predicting hemodynamic instability in Asian intensive care unit (ICU) patients.
• To assess the generalizability of the HSI model in an external cohort.

Main Methods:

• A retrospective study of 15,967 adult ICU patients at Taipei Veteran General Hospital (2010-2020).
• Hemodynamic instability defined by interventions (inotropic, vasopressor, fluid therapy, blood transfusions).
• HSI scores calculated hourly pre-intervention; model performance evaluated using AUROC, compared against Shock Index and Systolic Blood Pressure (SBP).

Main Results:

• The HSI achieved an AUROC of 0.76, significantly outperforming Shock Index (0.70) and SBP (0.69).
• With a threshold of 0.7, HSI predicted 72% of unstable patients with 67% specificity.
• HSI provided significant lead time, identifying 95% of unstable patients over 5 hours in advance, outperforming single indicators up to 24 hours prior.

Conclusions:

• The HSI demonstrates acceptable discrimination for predicting hemodynamic instability in this Asian cohort.
• The model shows potential for early risk identification, offering a significant lead time before interventions are needed.
• Further research is needed as the HSI may underestimate the risk of instability in patients who remain hemodynamically stable.