Global and local feature communications with transformers for 3D human pose estimation

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Spatiotemporal Transformer networks are state-of-the-art for 3D human pose estimation.
• Current methods often limit attention to individual joints within frames or trajectories, restricting global feature communication.

Purpose of the Study:

• To address the limitations of existing Transformer structures in 3D human pose estimation.
• To propose a novel architecture, GaLFormer, that integrates both local and global feature learning.

Main Methods:

• Introduced GaLFormer, a network combining local Transformer blocks (joint tokens) and global mixing Transformer blocks.
• The global blocks facilitate feature exchange across joints within a defined frame range, enforcing inductive bias.
• Alternately repeated local and global blocks capture correlations between joints, shapes, and trajectories.

Main Results:

• GaLFormer demonstrated superior or competitive performance on benchmark datasets.
• Evaluated on Human 3.6M, MPI-INF-3DHP, and HumanEva datasets.
• Achieved state-of-the-art results in 3D human pose estimation.

Conclusions:

• GaLFormer effectively integrates local and global information for improved 3D human pose estimation.
• The proposed architecture enhances feature representation by considering broader spatial-temporal contexts.
• This work advances the capabilities of Transformer-based models in human pose analysis.