Learning Temporal-Spatial Contextual Adaptation for Three-Dimensional Human Pose Estimation.

Area of Science:

• Computer Vision
• Machine Learning
• Robotics

Background:

• Three-dimensional human pose estimation from 2D videos is crucial for applications like human-robot interaction and virtual reality.
• Current methods often analyze spatial and temporal cues independently, overlooking their synergistic effects.

Purpose of the Study:

• To propose a novel 3D human pose estimation method that captures the synergistic influence of spatial-temporal cues.
• To enhance 3D pose inference by adaptively learning these combined effects.

Main Methods:

• Introduced the Dual-Adaptive Spatial-Temporal Former (DASTFormer) with attention-adaptive (AtA) and pure-adaptive (PuA) modes.
• Implemented an additional supervised training strategy using batch variance loss with a two-round parameter update.

Main Results:

• The DASTFormer adaptively learns spatial-temporal effects, improving 2D to 3D pose inference.
• The novel training method effectively explores encoding-pose relationships and mitigates batch size limitations.
• Achieved significant performance improvements over state-of-the-art methods on Human3.6 and HumanEVA datasets.

Conclusions:

• The proposed DASTFormer method effectively addresses the limitations of existing approaches by considering spatial-temporal synergy.
• The novel training strategy enhances model performance and training efficiency for transformer-based frameworks.