Development and Evaluation of an Automated Machine Learning Algorithm for In-Hospital Mortality Risk Adjustment Among...

Area of Science:

• Critical Care Medicine
• Health Informatics
• Machine Learning in Healthcare

Background:

• Risk adjustment algorithms are crucial for evaluating and enhancing intensive care unit (ICU) performance.
• Existing algorithms face low adoption rates due to high licensing, implementation, and data collection labor costs.
• The widespread use of electronic health records presents an opportunity for automated risk adjustment solutions.

Purpose of the Study:

• To develop and validate a retrospective, automated risk adjustment algorithm for in-hospital mortality among ICU patients.
• To create a tool that overcomes the key barriers hindering the adoption of current ICU risk adjustment methods.

Main Methods:

• Utilized modern machine learning techniques and open-source tools to develop a risk adjustment model.
• The study involved 131 ICUs across 53 hospitals, with a cohort of 237,173 ICU patients.
• Data were randomly split into training and validation sets for model development and performance assessment.

Main Results:

• The developed Risk of Inpatient Death score demonstrated excellent discrimination (AUC=0.94) and calibration (adjusted Brier score=52.8%) in the validation dataset.
• The final model incorporated seventeen features, combining clinical and administrative data elements.
• Performance favorably compared to commonly used, human-intensive mortality risk adjustment algorithms.

Conclusions:

• The automated Risk of Inpatient Death score addresses major adoption barriers like cost and labor intensity.
• This algorithm offers a potentially valuable tool for improving healthcare value in ICUs.
• Further external validation is recommended to confirm performance across different healthcare settings.