Real-world implementation and predictive performance of suicide prediction models: A systematic review and...

Area of Science:

• Clinical Informatics
• Artificial Intelligence in Healthcare
• Mental Health Research

Background:

• Machine learning (ML) models show potential for suicide risk prediction.
• Real-world performance and reasons for variation in ML suicide prediction models are not well understood.

Purpose of the Study:

• To systematically evaluate the real-world discriminative ability of statistical and ML suicide prediction models.
• To identify factors contributing to performance variations in these models.

Main Methods:

• Systematic literature search of studies on suicide prediction models in clinical settings.
• Random-effects meta-analyses to pool the area under the receiver operating characteristic curve (AUC).
• Exploratory meta-regression and sensitivity analyses to investigate heterogeneity.

Main Results:

• Nine studies with 20 evaluations were included, showing an overall pooled AUC of 0.849.
• Pooled AUCs for suicide attempts, ideation, and death were 0.865, 0.835, and 0.842, respectively.
• Extreme heterogeneity (I² = 99.9%) was observed, with no clear associations found for outcome type, evaluation timing, or model type.

Conclusions:

• Suicide prediction models exhibit good real-world discrimination but substantial performance variability.
• No clear evidence suggests outcome type, evaluation timing, or model type explains this heterogeneity.
• Emphasizes the need for rigorous local validation and implementation tailored to clinical workflows.