Benchmarking action recognition models for self-harm detection in studio and real-world datasets.

Area of Science:

• Computer Science
• Artificial Intelligence
• Clinical Psychology

Background:

• Automated systems for preventing patient self-harm in psychiatric settings are crucial but lack sufficient realistic training data.
• Existing datasets often fail to capture the diversity and complexity of behaviors observed in real clinical environments.

Purpose of the Study:

• To introduce a novel dataset for training and evaluating automated self-harm prevention systems.
• To benchmark state-of-the-art action recognition models on simulated and real-world patient behaviors.
• To analyze the performance gap between simulated and real-world data and identify limitations.

Main Methods:

• A new dataset comprising 1120 simulated cutting videos and 118 real-world videos (including picking and scratching) was created.
• Comprehensive benchmarking of convolution-based and transformer-based action recognition models was performed.
• Performance evaluation used 7-fold Leave-One-Out-Across-Subjects (LOAO) cross-validation on simulated data and direct testing on real-world data.

Main Results:

• The top model, VideoMAEv2, achieved an F1 score of 0.65 on simulated data but dropped to 0.61 on real-world data, indicating a significant 'sim-to-real' gap.
• Models struggled to generalize due to the uniform nature of simulated actions compared to diverse, occluded behaviors in authentic settings.
• Qualitative analysis revealed specific failure points in model generalization.

Conclusions:

• Current action recognition models exhibit limitations in generalizing from simulated to real-world patient self-harm behaviors.
• The developed dataset and benchmark provide a foundation for future research in this critical area.
• There is an urgent need for more diverse datasets and advanced AI approaches to enhance patient safety technologies.