Development and Prospective Validation of a Machine Learning-Based Risk of Readmission Model in a Large Military...

Area of Science:

• Health Informatics
• Machine Learning in Healthcare
• Clinical Predictive Modeling

Background:

• Thirty-day hospital readmissions are a key healthcare quality metric.
• Predictive models for readmission risk are crucial for targeted preventive strategies.
• Prospective validation of these models, especially in military hospitals, is rare.

Purpose of the Study:

• To develop and prospectively validate machine learning (ML) risk of readmission models for clinical staff at a military medical facility.
• To demonstrate collaboration between the U.S. Department of Defense's integrated health care system and a private company.

Main Methods:

• Evaluated multiple ML algorithms using retrospective data from Madigan Army Medical Center (MAMC) for 30-day all-cause readmissions.
• Validated the developed model prospectively on MAMC patient data.
• Revised and retrained the model with additional retrospective data after initial prospective evaluation.

Main Results:

• The initial retrospective model achieved an Area Under the Curve (AUC) of 0.68.
• The prospective model had an AUC of 0.64, with a 9.7% readmission rate among 1,574 patients.
• The revised model, incorporating more data, achieved an AUC of 0.76.

Conclusions:

• Operationalizing ML models in complex integrated health care systems requires significant collaborative effort.
• Prospective validation is essential for the reliable performance of ML-based predictive models in clinical settings.