Mutual Information Driven Equivariant Contrastive Learning for 3D Action Representation Learning.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Self-supervised contrastive learning is effective for skeleton-based action recognition.
• Traditional invariant contrastive learning can hinder performance when transformations contain task-relevant information.
• This limits the utility of many data transformations in current contrastive learning pipelines.

Purpose of the Study:

• To introduce equivariant contrastive learning as a method to preserve important information during data transformations.
• To develop a hybrid approach integrating invariant and equivariant contrastive learning for improved representation learning.
• To enhance skeleton-based action recognition by leveraging motion patterns more effectively.

Main Methods:

• Proposed a hybrid approach combining invariant and equivariant contrastive learning.
• Introduced a self-distillation loss for invariant transformations of varying intensities.
• Explored skeleton mixing and temporal shuffling for equivariant transformations.
• Developed novel metrics (consistency and diversity) to analyze feature space impacts.

Main Results:

• The hybrid model effectively leverages motion patterns from data transformations.
• Equivariant learning was shown to alleviate the dimensional collapse problem, boosting performance.
• Experimental results on multiple benchmarks demonstrated superior performance compared to existing methods.

Conclusions:

• The proposed equivariant contrastive learning approach significantly improves skeleton-based action recognition.
• The hybrid method offers a more discriminative representation space by preserving essential motion information.
• This work advances contrastive learning by addressing limitations of invariant transformations.