Skeleton-Based Spatio-Temporal U-Network for 3D Human Pose Estimation in Video.

Area of Science:

• Computer Vision
• Machine Learning
• Human Pose Estimation

Background:

• 3D pose estimation from videos is advancing, but extracting multi-scale spatio-temporal features from dynamic skeleton sequences remains challenging.
• Existing methods struggle with the complexity and granularity of human motion data.

Purpose of the Study:

• To propose a novel skeleton-based spatio-temporal U-Net (STUNet) for effective extraction of multi-scale spatio-temporal features.
• To address limitations in capturing fine-grained and coarse-grained features in dynamic skeleton sequences for 3D human pose estimation.

Main Methods:

• Developed a U-shaped network architecture (STUNet) combining semantic graph convolution and structural temporal dilated convolutions.
• Employed downscaling and upscaling for scale compression and feature squeezing.
• Utilized skip connections to abstract multi-resolution spatio-temporal dependencies.

Main Results:

• The STUNet model effectively captures comprehensive spatio-temporal features across multiple scales.
• Demonstrated substantial improvements over mainstream methods on real-world datasets for 3D human pose estimation.
• Validated the model's capability in handling complex dynamic skeleton sequences.

Conclusions:

• STUNet provides an effective solution for extracting multi-scale spatio-temporal features in 3D human pose estimation.
• The proposed architecture advances the state-of-the-art in analyzing complex dynamic skeleton data.
• This work offers a robust framework for future research in video-based human motion analysis.