PPG3D: Does 3D head tracking improve camera-based PPG estimation?

Area of Science:

• Biomedical Engineering
• Computer Vision
• Physiological Monitoring

Background:

• Camera-based vital sign estimation, or imaging photoplethysmography (iPPG), offers simple, unobtrusive measurements with potential applications in autonomous vehicles, neonatal care, and telemedicine.
• A key challenge for non-contact iPPG is mitigating motion artifacts, particularly relative motion between the camera and subject.
• Existing 2D feature tracking methods are limited to translational and in-plane motions, hindering iPPG robustness in real-world scenarios.

Purpose of the Study:

• To investigate the utility of 3D face tracking for robust iPPG estimation in the presence of significant out-of-plane and large relative motions.
• To overcome the limitations of 2D tracking methods in handling complex motion dynamics.
• To enhance the reliability and applicability of iPPG across diverse and dynamic environments.

Main Methods:

• Utilized an RGB-D camera to capture 3D facial information, including spatial and depth data.
• Developed a 3D face model fitting and tracking technique to maintain pixel-level correspondence across video frames, even with substantial motion.
• Applied iPPG estimation on motion-compensated (warped) video data to ensure accurate per-pixel correspondence over the estimation window.

Main Results:

• Demonstrated significantly improved robustness of iPPG estimation when subjected to large head motions.
• Achieved pixel-level correspondence accuracy even with out-of-plane and large relative motions.
• Validated the effectiveness of 3D face tracking in overcoming motion-related challenges in iPPG.

Conclusions:

• 3D face tracking is a viable and effective method for enhancing the robustness of imaging photoplethysmography (iPPG).
• This approach significantly improves iPPG performance in scenarios with substantial subject or camera motion.
• The findings pave the way for more reliable and widespread adoption of non-contact vital sign monitoring in complex environments.