Face-from-Depth for Head Pose Estimation on Depth Images.

Area of Science:

• Computer Vision
• Machine Learning
• Human-Computer Interaction

Background:

• Depth cameras offer robust solutions for people monitoring in challenging lighting conditions where RGB sensors fail.
• Accurate head and shoulder pose estimation is crucial for understanding human behavior and enabling advanced human-computer interaction.

Purpose of the Study:

• To develop a complete end-to-end framework for head and shoulder pose estimation using solely depth images.
• To improve pose estimation accuracy by incorporating a face hallucination module.

Main Methods:

• A Convolutional Neural Network (CNN) named POSEidon+ processes depth images to estimate 3D pose angles.
• A Face-from-Depth component utilizing a Deterministic Conditional Generative Adversarial Network (GAN) synthesizes facial information.
• The framework includes modules for head detection and localization.

Main Results:

• The proposed framework achieves state-of-the-art performance on public datasets (Biwi Kinect Head Pose, ICT-3DHP) and a new automotive dataset (Pandora).
• Real-time performance exceeding 30 frames per second was demonstrated.
• Hallucinating faces from depth data was empirically shown to enhance system performance.

Conclusions:

• The POSEidon+ framework provides a reliable, real-time solution for head and shoulder pose estimation using only depth data.
• This approach is particularly effective in environments with poor or unstable illumination.
• The integration of GAN-based face hallucination significantly boosts the accuracy of pose estimation.