Multiscale Spatio-Temporal Graph Neural Networks for 3D Skeleton-Based Motion Prediction.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Human pose estimation and prediction are crucial for understanding human motion.
• Existing methods often struggle with capturing complex spatio-temporal dynamics across various scales.
• Action-category-agnostic prediction remains a significant challenge in human motion analysis.

Purpose of the Study:

• To propose a novel multiscale spatio-temporal graph neural network (MST-GNN) for action-category-agnostic 3D human pose prediction.
• To develop a data-adaptive graph structure that captures motion-based relations at various scales.
• To enhance the accuracy and robustness of future human pose forecasting.

Main Methods:

• Developed a multiscale spatio-temporal graph to model motion relations at diverse spatial and temporal scales.
• Introduced a multiscale spatio-temporal graph computational unit (MST-GCU) for feature embedding and fusion across scales.
• Employed an encoder-decoder architecture with MST-GCUs for feature learning and a graph-based attention gate recurrent unit (GA-GRU) for pose generation.

Main Results:

• MST-GNN significantly outperforms state-of-the-art methods in both short-term and long-term 3D human pose prediction.
• Achieved superior performance on Human 3.6M, CMU Mocap, and 3DPW datasets, with notable reductions in mean angle errors.
• Demonstrated improvements of up to 11.84% in mean angle errors on the CMU Mocap dataset.

Conclusions:

• The proposed MST-GNN effectively predicts future 3D human poses in an action-category-agnostic manner.
• The data-adaptive multiscale graph structure is key to capturing complex motion dynamics.
• MST-GNN represents a significant advancement in the field of human motion prediction.