PoseMoE: Mixture-of-Experts Network for Monocular 3D Human Pose Estimation

Area of Science:

• Computer Vision
• Machine Learning
• Human Pose Estimation

Background:

• Monocular 3D human pose estimation relies heavily on lifting-based methods using 2D poses.
• These methods struggle with depth estimation due to entangled features and initial depth uncertainty.
• Existing approaches limit accuracy by combining uncertain depth with refined 2D pose information.

Purpose of the Study:

• To develop a novel approach for monocular 3D human pose estimation that overcomes limitations of current lifting-based methods.
• To disentangle the encoding of 2D pose and depth features for improved accuracy.
• To enhance feature representation through cross-expert knowledge aggregation.

Main Methods:

• Introduced PoseMoE, a Mixture-of-Experts network for monocular 3D pose estimation.
• Utilized specialized expert modules to refine 2D pose features and learn depth features independently.
• Implemented a cross-expert knowledge aggregation module for spatio-temporal feature enhancement via bidirectional mapping.

Main Results:

• PoseMoE effectively disentangles 2D pose and depth feature encoding, mitigating the negative impact of initial depth uncertainty.
• The cross-expert aggregation module enhances feature representation by integrating spatio-temporal context.
• PoseMoE demonstrated superior performance compared to conventional lifting-based methods on Human3.6M, MPI-INF-3DHP, and 3DPW datasets.

Conclusions:

• The proposed Mixture-of-Experts approach (PoseMoE) offers a significant advancement in monocular 3D human pose estimation.
• Disentangling 2D pose and depth feature learning is crucial for accurate depth estimation.
• PoseMoE provides a more robust and accurate solution for monocular 3D human pose estimation.