Transformer and Adaptive Threshold Sliding Window for Improving Violence Detection in Videos

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Video-based violence detection is crucial for public safety and content moderation.
• Existing models like Vision Transformers (ViT) show promise but struggle with cross-dataset generalization.
• Challenges include performance degradation when applied to unseen datasets.

Purpose of the Study:

• To develop a robust video violence detection system.
• To address the generalization limitations of current deep learning models.
• To enhance the accuracy and reliability of violence event detection across diverse datasets.

Main Methods:

• Developed CrimeNet, a Vision Transformer (ViT) model incorporating structured neural learning and adversarial regularization.
• Implemented an adaptive threshold sliding window model, also based on Transformer architecture, for post-processing.
• Evaluated performance on multiple benchmark datasets including XD-Violence, UCF-Crime, and RWF-2000.

Main Results:

• CrimeNet achieved high performance (AUC ROC up to 99%, AUC PR up to 100%) on individual datasets.
• Cross-dataset generalization issues caused a 20-30% performance drop.
• The adaptive threshold sliding window improved cross-dataset accuracy by 10-15%.

Conclusions:

• The combined approach of CrimeNet and the adaptive sliding window model significantly enhances video violence detection accuracy, especially in cross-dataset scenarios.
• This method effectively mitigates generalization problems inherent in deep learning models.
• Future work should focus on further improving generalization, handling data imbalance, and exploring multimodal approaches.