Motif-GCNs With Local and Non-Local Temporal Blocks for Skeleton-Based Action Recognition

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Graph convolutional networks (GCNs) excel at action recognition using skeletal data, but often rely on fixed graph topologies.
• Existing methods struggle to learn complex, sample-dependent joint relationships and fully exploit temporal features.
• Traditional convolutions with small kernels limit the capture of long-range dependencies in skeletal sequences.

Purpose of the Study:

• To propose a novel motif-based graph convolution method for enhanced action recognition.
• To address limitations in learning sample-dependent joint relationships and temporal feature extraction.
• To develop a model capable of capturing both local and non-local spatial-temporal dependencies.

Main Methods:

• Introduced a motif-based graph convolution that utilizes sample-dependent latent relations among non-physically connected joints.
• Developed a sparsity-promoting loss function to learn a sparse motif adjacency matrix for latent dependencies.
• Proposed efficient local and non-local temporal blocks to capture temporal information at different scales, integrated into sparse motif-based graph convolutional networks (SMotif-GCNs).

Main Results:

• The proposed SMotif-GCNs model demonstrated superior performance in human action recognition tasks.
• Achieved state-of-the-art results on four large-scale skeletal action recognition datasets.
• The method effectively captures both local and non-local spatial-temporal relationships.

Conclusions:

• The motif-based approach significantly improves action recognition accuracy by learning dynamic skeletal relationships.
• The integration of local and non-local temporal modeling enhances the model's ability to understand complex actions.
• The developed SMotif-GCNs offer a powerful new framework for skeletal-based action recognition.