Zero-Shot Skeleton-Based Action Recognition With Prototype-Guided Feature Alignment.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Zero-shot skeleton-based action recognition classifies unseen human actions using skeleton data.
• Existing methods struggle with insufficient skeleton feature discrimination and alignment bias.
• Two-stage training approaches often fail to generalize effectively to unknown actions.

Purpose of the Study:

• To propose a prototype-guided feature alignment (PGFA) paradigm for improved zero-shot skeleton-based action recognition.
• To enhance skeleton feature discrimination and address alignment bias in cross-modal feature alignment.
• To enable more accurate classification of previously unseen human actions.

Main Methods:

• Developed an end-to-end cross-modal contrastive training framework for skeleton-text alignment.
• Introduced a prototype-guided text feature alignment strategy to mitigate distribution discrepancies.
• The PGFA paradigm integrates these components for comprehensive zero-shot action recognition.

Main Results:

• PGFA demonstrated significant improvements over the state-of-the-art SMIE method.
• Achieved absolute accuracy gains of 22.96% on NTU-60, 12.53% on NTU-120, and 18.54% on PKU-MMD.
• Empirical evaluation validated the effectiveness of the proposed methods on benchmark datasets.

Conclusions:

• The proposed PGFA paradigm effectively addresses limitations in prior zero-shot action recognition methods.
• PGFA enhances skeleton feature representation and cross-modal alignment for better generalization.
• This approach offers a promising direction for recognizing unseen human actions from skeleton data.