Training and Interpreting Machine Learning Algorithms to Evaluate Fall Risk After Emergency Department Visits

Area of Science:

• Gerontology
• Health Informatics
• Machine Learning

Background:

• Machine learning is increasingly utilized for healthcare risk stratification.
• Effective predictive models require translation into actionable interventions to improve patient outcomes.
• This study explores automated risk stratification and referral interventions for older adults at risk of falls post-emergency department (ED) visits.

Purpose of the Study:

• To evaluate machine learning methodologies for a fall risk stratification algorithm using electronic health record (EHR) data.
• To estimate the impact of a fall risk intervention based on algorithm performance.
• To assess the clinical utility of fall risk prediction models in older adults.

Main Methods:

• Retrospective collection of EHR data at ED discharge.
• Development of algorithms (random forests, AdaBoost, regression) to predict 6-month fall-related return visits.
• Evaluation using area under the receiver operating characteristic curve (AUC) and clinical impact metrics (number needed to treat - NNT, referrals per week).

Main Results:

• The random forest model achieved an AUC of 0.78; regression models showed slightly lower performance.
• Models with similar AUC performance exhibited different clinical impacts when assessing NNT.
• The study highlights variations in real-world clinical utility despite comparable AUC scores.

Conclusions:

• Automated risk stratification and intervention can identify older adults at high risk for falls after ED visits.
• Translating predictive model performance into clinical impact metrics like NNT is crucial for intervention planning.
• Decision-makers can use these analyses to understand the trade-offs between referral numbers and NNT before implementing interventions.