Generalized Pose Decoupled Network for Unsupervised 3D Skeleton Sequence-Based Action Representation Learning.

Area of Science:

• Computer Vision
• Machine Learning
• Human-Computer Interaction

Background:

• Traditional 3D human action representation methods using recurrent neural networks (RNNs) primarily capture human shape, with limited success in representing motion dynamics.
• Existing methods struggle to differentiate actions based on subtle variations in motion direction or magnitude.

Purpose of the Study:

• To develop an improved unsupervised learning method for 3D human action representation that effectively captures both human shape and motion information.
• To address the limitations of existing methods in distinguishing actions based on motion direction and norm.

Main Methods:

• Introduced 'pose flow,' a handcrafted motion feature, to guide autoencoder reconstruction for better motion representation.
• Proposed an explicit pose decoupled flow network (PDF-E) using a multi-task learning framework to separately learn motion direction and norm.
• Developed a generalized PDF network (PDF-G) incorporating pose reconstruction as a constraint to learn both motion and shape information.

Main Results:

• The PDF-E network demonstrated improved ability to learn distinctive features by decoupling motion direction and norm.
• The PDF-G network achieved state-of-the-art performance on large-scale 3D action recognition datasets, including NTU RGB+D 60 and NTU RGB+D 120.
• The proposed methods significantly enhance the representation of human motion dynamics compared to traditional RNN-based approaches.

Conclusions:

• Decoupling motion direction and norm is crucial for effective 3D human action representation.
• The generalized PDF network (PDF-G) offers a robust and effective solution for unsupervised learning of human action representation, capturing both shape and motion.
• This work advances the field of 3D action recognition by providing a more discriminative feature learning approach.