Related Experiment Video
Updated: Jun 13, 2025
07:31
Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
Published on: May 15, 2020
7.0K
Transatlantic transferability and replicability of machine-learning algorithms to predict mental health crises
João Guerreiro1, Roger Garriga2,3, Toni Lozano Bagén2
1Koa Health, Barcelona, Spain. joao.leitaoguerreiro@koahealth.com.
NPJ Digital Medicine
|September 9, 2024
Summary
Machine learning models can predict mental health crises across different healthcare systems. This study demonstrates the successful transfer and replication of these predictive algorithms between UK and US Electronic Health Records (EHRs).
Area of Science:
- Healthcare Informatics
- Machine Learning in Medicine
- Clinical Predictive Modeling
Background:
- Widespread adoption of machine learning in healthcare is hindered by challenges in transferring predictive algorithms across diverse systems.
- Electronic Health Records (EHRs) contain critical data for developing predictive models but vary significantly between institutions.
- Predicting mental health crises requires robust algorithms that can generalize across different healthcare settings.
Purpose of the Study:
- To explore the impact of healthcare system and EHR differences on machine learning algorithms for predicting mental health crises.
- To evaluate the transferability and replicability of machine learning models trained on UK EHR data for use in a US healthcare system.
- To assess the performance of predictive models for mental health crises up to 28 days in advance.
Main Methods:
- Trained six machine learning models using features and methods from UK EHR data.
- Applied these models to predict mental health crises in 2907 patients at a US healthcare system (Rush University System for Health, 2018-2020).
- Evaluated model performance using Area Under the Receiver Operating Characteristic curve (AUROC).
Main Results:
- The best model, using US-specific structured and anonymized patient note features, achieved an AUROC of 0.837.
- A model originally trained on UK structured data, when transferred and tuned with US data, achieved an AUROC of 0.826.
- Demonstrated high performance in predicting mental health crises across different healthcare systems.
Conclusions:
- Transferring and replicating machine learning models for mental health crisis prediction across diverse hospital systems is feasible.
- Differences in healthcare systems and EHRs have a manageable impact on model performance.
- This research supports the broader implementation of machine learning in clinical practice for mental health.
