Human-Structure-Aware Token Position Embedding for Tokenized Pose Estimation

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Tokenized Pose Estimation (TPE) shows promise for lightweight Human Pose Estimation (HPE).
• Current TPE methods lack explicit human structure priors, hindering performance in challenging scenarios like occlusion and ambiguity.
• Human structure priors are crucial for robust HPE.

Purpose of the Study:

• To introduce a novel position embedding method, Human-structure-aware Token Position Embedding (HTPE), for TPE models.
• To enhance TPE by integrating human body structural information into keypoint and patch token embeddings.
• To improve the accuracy and robustness of lightweight HPE models.

Main Methods:

• Proposing Structure-Aware Keypoint Position Embedding (SAKPE) to encode human body symmetries and order into keypoint tokens.
• Developing Layer-adaptive Hybrid Patch Position Embedding (LHPPE) for adaptive fusion of absolute and relative patch token positions.
• Integrating SAKPE and LHPPE to form the HTPE method.

Main Results:

• HTPE significantly boosts the performance of various TPE models.
• Achieved state-of-the-art (SOTA) performance on COCO, CrowdPose, and OCHuman datasets among lightweight methods.
• Demonstrated consistent improvements under occlusion, with up to 3.3 AP gains, while minimally impacting parameters and FLOPs.

Conclusions:

• HTPE offers a significant advancement in lightweight human pose estimation.
• The proposed method effectively leverages human structural priors for improved accuracy and robustness.
• HTPE represents a new SOTA for efficient and accurate human pose estimation, particularly in occluded scenarios.