Attack-Augmented Mixing-Contrastive Skeletal Representation Learning.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Unsupervised action recognition relies on contrastive learning with positive and negative sample pairs.
• Existing methods use data augmentations for pair construction, but these often lack semantic variation and blur pair boundaries.
• Random appearance alterations in skeletons fail to capture meaningful semantic differences and weaken contrastive objectives.

Purpose of the Study:

• To develop an attack-driven augmentation framework for generating semantically meaningful perturbations in skeleton data.
• To create informative hard positive and hard negative samples for robust representation learning in unsupervised action recognition.
• To introduce a novel framework, Attack-Augmented Mixing-Contrastive Skeletal Representation Learning (A^2MC), for enhanced contrastive learning.

Main Methods:

• Proposed an Attack-Augmentation (Att-Aug) module integrating targeted (attack-based) and untargeted (augmentation-based) perturbations to generate hard positive samples.
• Introduced a Positive-Negative Mixer (PNM) to synthesize challenging hard negatives by blending hard positive and negative features.
• Implemented a mixed memory bank updated with synthesized hard negatives for improved contrastive learning.

Main Results:

• The A^2MC framework effectively generates semantically rich hard positive and hard negative samples.
• The proposed methods significantly enhance the robustness of skeleton representations for action recognition.
• Evaluations on three benchmarks show A^2MC performance is competitive with, or superior to, state-of-the-art methods.

Conclusions:

• Attack-driven augmentation provides a more effective strategy for creating informative sample pairs in contrastive learning.
• The A^2MC framework offers a novel and effective approach to unsupervised skeleton-based action recognition.
• The method demonstrates the potential of explicit semantic perturbations for advancing representation learning.