Importance of variables from different time frames for predicting self-harm using health system data

Area of Science:

• Biomedical Informatics
• Clinical Informatics
• Health Services Research

Background:

• Self-harm risk prediction models often utilize historical patient data spanning several years.
• Data availability across different time periods can be inconsistent for all patients.
• Algorithm-agnostic variable importance provides a framework to assess predictive potential across various time horizons.

Purpose of the Study:

• To evaluate the predictive potential of patient mental health information from different time horizons (recent vs. distant) for self-harm risk.
• To demonstrate the application of variable importance techniques in a biomedical informatics context for risk prediction.
• To understand how data availability at different time frames impacts model implementation.

Main Methods:

• Utilized variable importance to quantify the predictive power of recent (≤3 months) and distant (>1 year) mental health data.
• Assessed importance by measuring the decrease in predictiveness when specific variable sets were excluded.
• Employed discriminative metrics including area under the receiver operating characteristic curve (AUC), sensitivity, and positive predictive value.

Main Results:

• Mental health predictors from the three months preceding the index visit demonstrated significant importance.
• Excluding recent predictors reduced the area under the receiver operating characteristic curve (AUC) from 0.85 to 0.77 in one setting.
• Predictors from more distant time frames showed comparatively lower importance.

Conclusions:

• Recent mental health indicators are highly important for accurate self-harm risk prediction.
• Challenges in implementing self-harm prediction models may arise in settings with incomplete recent data due to processing lags.
• Variable importance analysis is valuable for guiding the clinical implementation of risk prediction models amidst data limitations and can be broadly applied in biomedical informatics.