DroneAttention: Sparse weighted temporal attention for drone-camera based activity recognition.

Area of Science:

• Computer Vision
• Artificial Intelligence

Background:

• Human activity recognition (HAR) using drone-mounted cameras is crucial for surveillance, crowd analysis, and human-computer interaction.
• Challenges include complex poses, varying viewpoints, and diverse environmental conditions.

Purpose of the Study:

• To propose a novel Sparse Weighted Temporal Attention (SWTA) module for enhanced HAR from drone imagery.
• To improve the efficiency and accuracy of HAR systems by utilizing sparsely sampled frames.

Main Methods:

• Developed a two-part SWTA module: a temporal segment network for sparse sampling and weighted temporal attention fusing optical flow and RGB data.
• Integrated SWTA as a plug-in module with existing deep Convolutional Neural Network (CNN) architectures.
• Evaluated the model on Okutama, MOD20, and Drone-Action benchmark datasets.

Main Results:

• Achieved high accuracy: 72.76% on Okutama, 92.56% on MOD20, and 78.86% on Drone-Action.
• Significantly surpassed previous state-of-the-art performances by margins of 25.26%, 18.56%, and 2.94% respectively.
• Demonstrated the module's effectiveness in learning temporal information without a separate temporal stream.

Conclusions:

• The proposed SWTA module offers a robust and efficient solution for drone-based HAR.
• SWTA effectively addresses complexities in pose, viewpoint, and environmental scenarios.
• This approach optimizes existing CNNs for temporal learning, advancing the field of computer vision.