Multi-View Time-Series Hypergraph Neural Network for Action Recognition

Summary

This summary is machine-generated.

This study introduces a novel Multi-View Time-Series Hypergraph Neural Network (MV-TSHGNN) to enhance skeleton-based action recognition. The method significantly improves accuracy in complex scenarios by capturing high-order relationships in spatial and temporal data.

Area Of Science

  • Computer Vision
  • Machine Learning
  • Artificial Intelligence

Background

  • Skeleton-based action recognition faces challenges like occlusion, poor lighting, and weak joint correlations, leading to low accuracy.
  • Existing methods struggle with complex dynamic environments and intricate human body joint relationships.

Purpose Of The Study

  • To propose a novel Multi-View Time-Series Hypergraph Neural Network (MV-TSHGNN) for improving skeleton-based human action recognition.
  • To address the limitations of current methods in dynamic and complex environments.

Main Methods

  • The MV-TSHGNN framework constructs multi-view time-series hypergraph structures and employs hypergraph convolutions.
  • It extracts joint features from different views, constructs spatial hypergraphs for limb components and adjacent joints, and temporal hypergraphs for continuous joint movements within views.
  • A multi-view time-series hypergraph neural network learns spatial and temporal features to capture high-order semantic relationships.

Main Results

  • The MV-TSHGNN method achieved state-of-the-art performance on benchmark datasets including NTU RGB+D, NTU RGB+D 120, and imitating traffic police gestures.
  • Experimental results demonstrate the effectiveness and efficiency of the proposed model in complex action recognition tasks.

Conclusions

  • The proposed MV-TSHGNN effectively improves the accuracy of skeleton-based action recognition by leveraging multi-view spatial-temporal hypergraph learning.
  • The method offers a promising approach for robust action recognition in challenging real-world conditions.