Reformulating Graph Kernels for Self-Supervised Space-Time Correspondence Learning.

Area of Science:

• Computer Vision
• Machine Learning
• Graph Theory

Background:

• Self-supervised learning for space-time correspondence in unlabeled videos is crucial for computer vision.
• Existing methods often require dense frame affinities or optical flow, limiting their applicability.
• Video correspondence models need to capture inherent structural properties for robust performance.

Purpose of the Study:

• To propose HiGraph+, a novel self-supervised framework for learning space-time correspondences in videos.
• To leverage learnable graph kernels for predicting hidden spatial-temporal graphs.
• To enhance the understanding of inherent video properties like structural information.

Main Methods:

• Videos are modeled as spatial-temporal graphs, with learning objectives derived from predicting hidden graphs using graph kernel methods.
• Graph-level correspondence learning focuses on structural consistency of sub-graphs.
• A spatio-temporal hidden graph loss using contrastive learning is introduced for temporal coherence and spatial diversity.

Main Results:

• HiGraph+ successfully predicts long-term correspondences by learning distinct local structural representations.
• Node-level representations are refined across frames using dense graph kernels.
• The framework demonstrates robustness and excellent performance on benchmark tasks including object, semantic part, keypoint, and instance labeling propagation.

Conclusions:

• The proposed HiGraph+ framework effectively utilizes self-supervision through graph structural and temporal consistency.
• The method advances self-supervised space-time correspondence learning by incorporating graph-based approaches.
• Publicly available implementation facilitates further research and application in computer vision.