Related Experiment Video
Updated: Jun 6, 2026
Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
Published on: May 15, 2020
Real-world implementation and predictive performance of suicide prediction models: A systematic review and
KangHyun Kim1, JuHee Kim1, Myung-Gwan Kim1
1Department of Biomedical Informatics, CHA University School of Medicine, Seongnam, South Korea.
Background:
Machine learning (ML) models for suicide risk prediction show promise, but their real-world discriminative ability and the sources of performance variation remain unclear.
Methods:
We systematically searched databases for studies evaluating statistical or ML suicide prediction models in real-world clinical settings. Random-effects meta-analyses pooled the area under the receiver operating characteristic curve (AUC) overall and by outcome type. We conducted exploratory univariable meta-regression with cluster-robust inference for outcome type, evaluation timing, and model type. Sensitivity analyses assessed the influence of within-study dependence.
Results:
Nine studies (20 evaluations) were included. The overall pooled AUC was 0.849 (95% CI 0.827-0.869). Pooled AUCs were 0.865 (0.840-0.887) for suicide attempts, 0.835 (0.825-0.844) for suicidal ideation, and 0.842 (0.797-0.878) for suicide death. Heterogeneity was extreme (I2 = 99.9%). Sensitivity analyses yielded similar pooled estimates. Exploratory univariable meta-regression showed no clear associations for outcome type, evaluation timing, or model type.
Conclusions:
Suicide prediction models show good real-world discrimination, but performance varies substantially across evaluations. We found no clear evidence that outcome type, evaluation timing, or model type explained this heterogeneity. These findings support rigorous local validation and workflow-sensitive implementation rather than assuming that more complex algorithms will perform better in practice.
