A Novel Unsupervised Video Anomaly Detection Framework Based on Optical Flow Reconstruction and Erased Frame...

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Traditional video anomaly detection (VAD) methods struggle to leverage contextual information for accurate anomaly perception.
• Reconstruction-based and prediction-based approaches are common but limited in smart city surveillance.

Purpose of the Study:

• To develop a novel unsupervised learning framework for video anomaly detection (VAD).
• To effectively encode both motion and appearance information at an object level using a Cloze Test strategy.
• To improve the accuracy and reliability of anomaly detection in smart city surveillance.

Main Methods:

• An optical stream memory network with skip connections was designed for normal video activity reconstruction.
• A space-time cube (STC) was used as the basic processing unit, with patches erased to form incomplete events (IEs) for reconstruction.
• A conditional autoencoder and a generative adversarial network (GAN)-based training method were employed to predict erased patches and enhance VAD performance.

Main Results:

• The proposed method achieved high Area Under the Receiver Operating Characteristic (AUROC) scores on benchmark datasets: 97.7% on UCSD Ped2, 89.7% on CUHK Avenue, and 75.8% on ShanghaiTech.
• The framework effectively captures high correspondence between optical flow and STC, enabling reliable anomaly detection.
• Distinguishing predicted erased optical flow and video frames improved the reliability of VAD results.

Conclusions:

• The novel unsupervised learning framework, inspired by NLP's Cloze Test, significantly enhances video anomaly detection.
• The method's ability to reconstruct incomplete events by predicting erased patches improves the utilization of contextual information.
• This approach offers a more reliable solution for anomaly detection in smart city surveillance applications.