Skeleton motion topology-masked prediction and contrastive learning for self-supervised human action recognition

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Self-supervised human action recognition faces challenges with data augmentation and joint dependency modeling.
• Existing methods may overemphasize salient motion regions or local joint trajectories.

Purpose of the Study:

• To propose a novel hybrid framework integrating topology-masked motion modeling and contrastive learning for enhanced self-supervised human action recognition.
• To address limitations in data augmentation and joint dependency neglect.

Main Methods:

• Developed a motion topology-masking technique encoding skeletal topology and motion dynamics.
• Employed a multi-stage hybrid augmentation strategy for diverse positive pairs in contrastive learning.
• Introduced a trajectory-guided feature dropping module to prevent over-focus on local joint trajectories.

Main Results:

• Achieved superior performance on NTU-60, NTU-120, and PKU-MMD datasets.
• Demonstrated significant improvements in occluded scenarios and low-supervision conditions.
• Effectively mitigated visual interference and annotation scarcity.

Conclusions:

• The proposed framework leverages large-scale unlabeled skeleton data effectively through self-supervised learning.
• Significantly reduces reliance on costly annotated datasets for human action recognition.
• Offers a robust solution for accurate action recognition amidst complex environments and data limitations.