Contextual information based anomaly detection for multi-scene aerial videos

Aerial video surveillance using Unmanned Aerial Vehicles (UAV) is gaining much interest worldwide due to its extensive applications in monitoring wildlife, urban planning, disaster management, anomaly detection, campus security, etc. These videos are processed and analyzed for strange/odd/anomalous patterns, which are essential requirements of surveillance. But manual analysis of these videos is tedious, subjective, and laborious. Hence, developing computer-aided systems for analyzing UAV-based surveillance videos is crucial. Despite this interest, in the literature, most of the video surveillance applications are developed focusing only on CCTV-based surveillance videos which are static. Thus, these methods cannot be extended for scenarios where the background/context information is dynamic (multi-scene). Further, the lack of standard UAV-based anomaly detection datasets has restricted the development of novel algorithms. In this regard, the present work proposes a novel multi-scene aerial video anomaly detection dataset with frame-level annotations. In addition, a novel Computer Aided Decision (CAD) support system is proposed to analyze and detect anomalous patterns from UAV-based surveillance videos. The proposed system holistically utilizes contextual, temporal, and appearance features for the accurate detection of anomalies. A novel feature descriptor is designed to effectively capture contextual information necessary for analyzing multi-scene videos. Additionally, temporal and appearance features are extracted to handle the complexities of dynamic videos, enabling the system to recognize motion patterns and visual inconsistencies over time. Furthermore, a new inference strategy is proposed that utilizes a few anomalous samples along with normal samples to identify better decision boundaries. The proposed method is extensively evaluated on the proposed UAV anomaly detection dataset and performs competitively with respect to state-of-the-art methods with an AUC of 0.712.