Multimodal and multiscale feature fusion for weakly supervised video anomaly detection.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Weakly supervised video anomaly detection uses video-level labels without segment boundaries.
• Existing methods often employ multiple instance learning but struggle with video blur and occlusion.
• These limitations hinder accurate detection of anomalous events.

Purpose of the Study:

• To develop a novel weakly supervised video anomaly detection method.
• To address challenges posed by video blur and visual occlusion.
• To improve the accuracy and robustness of anomaly detection systems.

Main Methods:

• Utilized pre-trained I3D for extracting RGB and optical flow features (appearance and motion).
• Introduced an Attention De-redundancy (AD) module to filter irrelevant features.
• Developed a Multi-scale Feature Learning module for temporal dependencies.
• Implemented an Adaptive Feature Fusion module for optimal feature integration.

Main Results:

• The proposed method significantly outperforms existing unsupervised and weakly supervised approaches.
• Achieved 97.00% AUC on the ShanghaiTech dataset.
• Achieved 85.31% AUC on the UCF-Crime dataset.

Conclusions:

• The fusion of multimodal and multiscale features enhances video anomaly detection.
• The novel AD and adaptive fusion modules effectively address feature redundancy and modality integration.
• The method demonstrates strong generalization and robustness on benchmark datasets.