Predicting Momentary Suicidal Ideation From Smartphone Screenshots Using Vision-Language Models: Prospective Machine...

Area of Science:

• Digital Health
• Artificial Intelligence in Mental Health
• Computational Psychiatry

Background:

• Passive smartphone sensing offers potential for suicide prevention but lacks contextual data for acute distress detection.
• Analyzing on-phone content (vision, text) may provide more direct indicators of psychological risk than usage patterns alone.

Purpose of the Study:

• To evaluate the efficacy of vision-language models (VLMs) in predicting momentary suicidal ideation (SI) from smartphone screenshots.
• To compare the performance of VLM-based prediction against text-only models and traditional lexical screening.

Main Methods:

• Seventy-nine adults with recent suicidal thoughts/behaviors underwent 28-day monitoring with passive screenshot capture.
• Fine-tuned open-source VLMs and text-only models to predict SI from screenshots preceding ecological momentary assessments (EMAs).
• Evaluated model performance using temporal and subject holdout validation strategies.

Main Results:

• VLM-based models demonstrated strong discrimination (AUC=0.83) at the EMA level, outperforming text-only models.
• Subject-level generalization was limited (AUC≈0.50), indicating a need for personalization.
• Smaller models showed comparable performance, suggesting feasibility for on-device deployment.

Conclusions:

• Smartphone screen content analysis effectively predicts short-term SI with personalized models.
• A two-stage clinical approach is proposed: initial lexical screening followed by personalized VLM monitoring.
• On-device inference holds potential for privacy-preserving mental health monitoring.