CNN-ViT Supported Weakly-Supervised Video Segment Level Anomaly Detection.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Video anomaly event detection (VAED) is crucial for smart surveillance.
• Deep learning has advanced VAED, with weakly supervised VAED (WVAED) gaining traction.
• Current WVAED methods rely heavily on pretrained feature extractors.

Purpose of the Study:

• To develop a robust WVAED method that leverages diverse pretrained feature extractors.
• To effectively capture both long-range and short-range temporal dependencies in video data.
• To propose a generalized architecture for WVAED that integrates multiple feature extraction techniques.

Main Methods:

• Utilized pretrained Convolutional Neural Network (CNN) models (C3D, I3D) and Vision Transformer (ViT) models (CLIP) for feature extraction.
• Introduced a Temporal Self-Attention Network (TSAN) to model temporal dependencies.
• Designed a Multiple Instance Learning (MIL)-based architecture, CNN-ViT-TSAN, integrating CNN/ViT features and TSAN.

Main Results:

• The proposed CNN-ViT-TSAN architecture demonstrated effectiveness in WVAED.
• The method successfully extracted discerning representations using combined feature extractors.
• Experimental results on crowd datasets validated the approach's performance.

Conclusions:

• The CNN-ViT-TSAN architecture offers a promising approach for weakly supervised video anomaly event detection.
• Integrating diverse feature extractors and temporal attention mechanisms enhances WVAED performance.
• The proposed method provides a generalized framework for tackling WVAED challenges.