Action Recognition Based on Multi-Level Topological Channel Attention of Human Skeleton

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Skeleton data is crucial for action recognition, mitigating environmental noise like background and lighting variations.
• Existing Graph Convolutional Networks (GCNs) struggle to fully exploit human body structure and limb coordination knowledge.

Purpose of the Study:

• To propose a novel Multi-level Topological Channel Attention Network (MTCA-Net) for improved human action recognition.
• To enhance feature extraction by integrating human body topology and limb coordination priors.
• To capture multi-scale spatio-temporal features effectively.

Main Methods:

• The Multi-level Topology and Channel Attention Module integrates coarse-to-fine human body structure knowledge.
• The Coordination Module utilizes contralateral and ipsilateral limb movements for kinematic analysis.
• A Multi-scale Global Spatio-temporal Attention Module captures features at different granularities, using causal convolutions and masked temporal attention.

Main Results:

• Achieved 91.9% accuracy on NTU-RGB+D 60 (Xsub) and 96.3% (Xview).
• Achieved 88.5% accuracy on NTU-RGB+D 120 (Xsub) and 90.3% (Xset).
• Demonstrated superior performance in action recognition tasks.

Conclusions:

• The proposed MTCA-Net effectively utilizes human body structure and limb coordination for robust action recognition.
• The network's multi-level attention mechanisms capture complex spatio-temporal dynamics.
• The method significantly advances the state-of-the-art in skeleton-based action recognition.