Self-Supervised Video-Based Action Recognition With Disturbances.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Self-supervised video-based action recognition requires extracting key action information from diverse, unlabeled videos.
• Existing methods often prioritize visual spatio-temporal features, neglecting semantic information crucial for human-like cognition.
• Action recognition is challenged by variations in actors, scenes, and semantic encoding.

Purpose of the Study:

• To propose VARD (Video-based Action Recognition with Disturbances), a novel self-supervised method for robust video action recognition.
• To extract principal action information by integrating both visual and semantic attributes, inspired by human cognitive processes.
• To develop a method that focuses on essential action characteristics by minimizing the impact of inconsequential visual and semantic variations.

Main Methods:

• VARD constructs a 'positive' clip/embedding for each action video, which is visually/semantically disturbed compared to the original.
• The method aims to minimize the distance between the original and disturbed representations in the latent space.
• It leverages Video Disturbance and Embedding Disturbance techniques to achieve this objective.
• Notably, VARD does not require optical flow, negative samples, or pretext tasks.

Main Results:

• VARD effectively improves upon strong baselines in self-supervised action recognition.
• The method demonstrates superior performance compared to multiple classical and advanced self-supervised techniques.
• Experiments were validated on the widely-used UCF101 and HMDB51 datasets.

Conclusions:

• VARD offers an effective approach to self-supervised video-based action recognition by integrating visual and semantic information.
• The proposed disturbance-based method successfully drives networks to focus on principal action information, enhancing robustness.
• VARD presents a simplified yet powerful alternative to existing methods, achieving state-of-the-art results without complex requirements.