H₂OT: Hierarchical Hourglass Tokenizer for Efficient Video Pose Transformers.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Transformers are effective for video-based 3D human pose estimation.
• High computational costs of current video pose transformers (VPTs) limit their use on devices with limited resources.

Purpose of the Study:

• To develop an efficient, plug-and-play framework for transformer-based 3D human pose estimation from videos.
• To address the computational inefficiency of existing VPT models.

Main Methods:

• Proposed Hierarchical Hourglass Tokenizer (H2OT) framework with Token Pruning Module (TPM) and Token Recovering Module (TRM).
• TPM dynamically selects representative pose tokens from redundant video frames.
• TRM restores spatio-temporal information from selected tokens for full-length sequence inference.

Main Results:

• H2OT significantly improves model efficiency by reducing intermediate transformer blocks' pose tokens.
• The method is general-purpose, compatible with various VPT models and pipelines (seq2seq, seq2frame).
• Achieved high efficiency and accuracy in 3D human pose estimation on benchmark datasets.

Conclusions:

• Maintaining full pose sequences is unnecessary for efficient and accurate 3D human pose estimation.
• H2OT offers a practical solution for efficient transformer-based video pose estimation on resource-constrained devices.