STTG-net: a Spatio-temporal network for human motion prediction based on transformer and graph convolution network.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Robotics

Background:

• Human motion prediction is complex due to motion's stochastic nature.
• Previous methods primarily focused on temporal dependencies, neglecting spatial joint correlations.

Purpose of the Study:

• To propose a novel spatio-temporal network (STTG-Net) for enhanced human motion prediction.
• To address limitations of existing methods, such as error accumulation and discontinuity.

Main Methods:

• Developed STTG-Net integrating a temporal transformer for global dependencies and a graph convolutional network (GCN) for local spatial correlations.
• Implemented a fusion strategy to refine predictions by integrating current and previous frames, mitigating error accumulation.

Main Results:

• The proposed STTG-Net demonstrated significantly reduced prediction error compared to prior methods.
• Generated smoother human motion predictions, enhancing visual realism and continuity.
• Achieved state-of-the-art performance on the Human3.6M dataset.

Conclusions:

• The STTG-Net effectively models spatio-temporal joint couplings for accurate human motion prediction.
• The fusion strategy successfully improves prediction continuity and reduces cumulative errors.
• This approach offers a promising advancement in realistic human motion synthesis and analysis.