Attention-Based 3D Human Pose Sequence Refinement Network.

Area of Science:

• Computer Vision
• Deep Learning
• 3D Human Pose Estimation

Background:

• Deep learning has advanced 3D human mesh reconstruction from single videos.
• Existing methods often produce temporally noisy pose and mesh sequences from in-the-wild videos.

Purpose of the Study:

• To develop a novel human pose refinement network (HPR-Net) to address temporal noise in 3D human mesh reconstruction.
• To improve the accuracy and temporal smoothness of reconstructed 3D human mesh sequences.

Main Methods:

• Proposes HPR-Net, a post-processing network utilizing a non-local attention mechanism.
• The framework includes weight-regression, weighted-averaging, and a skinned multi-person linear (SMPL) module.
• Pose affinity weights are generated from 3D pose sequences in unit quaternion form for temporal weighted averaging.

Main Results:

• HPR-Net substantially improves accuracy and temporal smoothness of existing 3D human mesh reconstruction methods.
• Demonstrates consistent improvement on various real-world datasets.
• Reduces pose and acceleration errors of VIBE by 1.4% and 66.5% on the 3DPW dataset, respectively.

Conclusions:

• HPR-Net is an effective post-processing solution for enhancing 3D human mesh reconstruction from videos.
• The proposed method significantly enhances temporal consistency and accuracy in human motion capture.
• Achieves state-of-the-art performance improvements on benchmark datasets.