Multi-View Time-Series Hypergraph Neural Network for Action Recognition.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Skeleton-based action recognition faces challenges like occlusion, poor lighting, and weak joint correlations, leading to low accuracy.
• Existing methods struggle with complex dynamic environments and intricate human body joint relationships.

Purpose of the Study:

• To propose a novel Multi-View Time-Series Hypergraph Neural Network (MV-TSHGNN) for improving skeleton-based human action recognition.
• To address the limitations of current methods in dynamic and complex environments.

Main Methods:

• The MV-TSHGNN framework constructs multi-view time-series hypergraph structures and employs hypergraph convolutions.
• It extracts joint features from different views, constructs spatial hypergraphs for limb components and adjacent joints, and temporal hypergraphs for continuous joint movements within views.
• A multi-view time-series hypergraph neural network learns spatial and temporal features to capture high-order semantic relationships.

Main Results:

• The MV-TSHGNN method achieved state-of-the-art performance on benchmark datasets including NTU RGB+D, NTU RGB+D 120, and imitating traffic police gestures.
• Experimental results demonstrate the effectiveness and efficiency of the proposed model in complex action recognition tasks.

Conclusions:

• The proposed MV-TSHGNN effectively improves the accuracy of skeleton-based action recognition by leveraging multi-view spatial-temporal hypergraph learning.
• The method offers a promising approach for robust action recognition in challenging real-world conditions.