Memory Attention Networks for Skeleton-Based Action Recognition.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Skeleton-based action recognition is challenging due to complex 3D spatiotemporal joint variations.
• Existing methods struggle to effectively model these intricate variations.

Purpose of the Study:

• To propose a novel method, Memory Attention Networks (MANs), for robust skeleton-based action recognition.
• To enhance MANs with a collaborative memory fusion module (CMFM) for improved efficiency, creating Collaborative MANs (C-MANs).

Main Methods:

• MANs utilize a temporal-then-spatial recalibration approach with a Temporal Attention Recalibration Module (TARM) and Spatiotemporal Convolution Module (STCM).
• TARM employs residual learning for temporal frame recalibration.
• STCM encodes recalibrated sequences for CNN-based spatiotemporal modeling.
• C-MANs integrate TARM, STCM, and CMFM for end-to-end training.

Main Results:

• MANs and C-MANs significantly outperform state-of-the-art methods in skeleton-based action recognition.
• The proposed methods achieved top performance across six benchmark datasets.
• The integrated network architecture enables seamless end-to-end training.

Conclusions:

• MANs and C-MANs offer a powerful and efficient solution for skeleton-based action recognition.
• The novel temporal-then-spatial recalibration and collaborative memory fusion effectively handle complex skeleton variations.
• The publicly available source code facilitates further research and application.