SelfPose: 3D Egocentric Pose Estimation From a Headset Mounted Camera.

Area of Science:

• Computer Vision
• Machine Learning
• Human-Computer Interaction

Background:

• Egocentric 3D body pose estimation from monocular images presents challenges due to unusual viewpoints, self-occlusions, and perspective distortions.
• Existing methods struggle with the drastic resolution differences between the upper and lower body in egocentric views.

Purpose of the Study:

• To develop an accurate and robust solution for egocentric 3D body pose estimation from downward-looking fisheye camera images.
• To address the scarcity of labeled training data for this specific viewpoint.

Main Methods:

• A novel encoder-decoder architecture with a multi-branch decoder was designed to handle varying uncertainty in 2D joint locations.
• A large-scale, photo-realistic synthetic dataset (xR-EgoPose) with 383K frames was created to facilitate training.

Main Results:

• The proposed method significantly improves accuracy over state-of-the-art egocentric pose estimation techniques on both synthetic and real-world data.
• The synthetic dataset's variability ensures good generalization to real-world footage, achieving state-of-the-art results on real datasets.
• Performance on the Human3.6M benchmark is comparable to top methods for traditional third-person viewpoint pose estimation.

Conclusions:

• The novel architecture and synthetic dataset effectively overcome the challenges of egocentric 3D body pose estimation from fisheye cameras.
• The approach demonstrates strong generalization capabilities and achieves competitive performance against established methods in human pose estimation.