Continual learning of conjugated visual representations through higher-order motion flows.

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• Continuous visual data streams pose challenges for neural networks due to non-i.i.d. data.
• Unsupervised continual learning aims to develop robust representations from evolving data.
• Existing methods often rely on external motion signals, limiting autonomous learning.

Purpose of the Study:

• To investigate unsupervised continual learning of pixel-wise features using motion-induced constraints.
• To develop motion-conjugated feature representations that are consistent with visual information flow.
• To enable autonomous learning of multi-order motion flows within the feature hierarchy.

Main Methods:

• Proposed a novel approach for unsupervised continual learning of visual features.
• Introduced motion-conjugated feature representations, learning motion autonomously at multiple levels.
• Developed a self-supervised contrastive loss, spatially-aware and flow-induced, to prevent trivial solutions.
• Estimated multiple motion flows, from optical flow to higher-order latent signals.

Main Results:

• The model demonstrated significant performance improvements on photorealistic synthetic streams and real-world videos.
• Achieved superior results compared to pre-trained state-of-the-art feature extractors (including Transformers).
• Outperformed recent unsupervised learning models in unsupervised continual learning tasks.

Conclusions:

• The proposed method effectively learns consistent multi-order flows and representations in a continual learning setting.
• Autonomous learning of motion-induced features offers a promising direction for visual representation learning.
• Motion-conjugated feature representations provide a robust solution for processing continuous visual data streams.