Unsupervised Video Anomaly Detection Based on Similarity with Predefined Text Descriptions

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Traditional video anomaly detection relies heavily on video data.
• Real-world applications benefit from incorporating human domain knowledge, often expressed as text.
• Unlabeled video datasets are abundant, presenting an opportunity for unsupervised learning.

Purpose of the Study:

• To explore the use of text descriptions for unsupervised video anomaly detection.
• To develop a method that leverages text to identify abnormal situations in videos without prior labeling.
• To improve anomaly detection performance by integrating textual domain knowledge with visual data.

Main Methods:

• Utilized large language models to generate text descriptions for video content.
• Employed the CLIP (Contrastive Language-Image Pre-training) visual language model to compute cosine similarity between video frames and text descriptions.
• Introduced a text-conditional similarity refinement using an unlabeled dataset and a triplet loss function.

Main Results:

• The proposed method significantly outperforms existing unsupervised methods on the ShanghaiTech and UCFcrime datasets, achieving 8% and 13% higher AUC scores, respectively.
• Demonstrated comparable performance to weakly supervised methods in detecting abnormal videos, despite not requiring manual labeling.
• Achieved 17% and 5% better AUC scores on abnormal videos compared to weakly supervised approaches, highlighting its effectiveness.

Conclusions:

• Text descriptions can be effectively integrated into unsupervised video anomaly detection frameworks.
• The proposed method offers a computationally efficient alternative to traditional approaches, avoiding optical flow or multi-frame analysis.
• This research validates the potential of using readily available text descriptions to enhance unsupervised anomaly detection in video data.