Cross-view motion consistent self-supervised video inter-intra contrastive for action representation understanding.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Self-supervised contrastive learning excels at extracting semantic features from unlabeled data.
• Existing video contrastive methods often overemphasize static backgrounds, hindering motion feature learning.
• Background shortcuts limit the effectiveness of current models in capturing dynamic visual information.

Purpose of the Study:

• To develop a self-supervised video contrastive model that accurately tracks motion features.
• To overcome the limitations of previous methods that focus on static backgrounds.
• To improve the learning of local details and long-term temporal relationships in videos.

Main Methods:

• Proposed a cross-view motion consistent (CVMC) self-supervised video inter-intra contrastive model.
• Extracted dynamic features from consecutive video snippets and aligned them using multi-view motion consistency.
• Optimized dynamic features for instance comparison across videos and fine-grained spatial-temporal analysis within videos.

Main Results:

• The CVMC model effectively learns visual representation information from unlabeled video data.
• Achieved highly competitive performance in action recognition tasks.
• Demonstrated strong results in video retrieval tasks compared to state-of-the-art methods.

Conclusions:

• The proposed joint optimization of spatio-temporal alignment and motion discrimination addresses key challenges in self-supervised learning.
• CVMC effectively improves instance recognition, spatial compactness, and temporal perception.
• This approach offers a robust method for learning from video data without manual labels.