Deep learning-based remote-photoplethysmography measurement from short-time facial video

Area of Science:

• Biomedical Engineering
• Computer Vision
• Signal Processing

Background:

• Non-contact heart rate (HR) measurement from facial video is crucial for health monitoring.
• Previous remote photoplethysmography (rPPG) methods often rely on manual feature engineering and unproven hypotheses.
• Challenges include complex environments, low-resolution video, and illumination variations.

Purpose of the Study:

• To develop an efficient, end-to-end framework for short-time HR estimation from facial videos.
• To extract robust rPPG signals without prior knowledge or manual ROI selection.
• To improve the accuracy and reduce the frame requirement for non-contact HR monitoring.

Main Methods:

• A deep 3D multi-scale network with a cross-layer residual structure autoencoder extracts rPPG features.
• A spatial-temporal fusion mechanism focuses on relevant rPPG signal features.
• Feature distillation and a data augmentation strategy address complex environments and data distribution issues.

Main Results:

• The proposed method outperforms state-of-the-art techniques on four benchmark datasets.
• Significant improvements in root mean square error (RMSE) were observed across different datasets (e.g., 5.9% on OBF, 3.4% on COHFACE, 21.4% on UBFC).
• The method requires fewer video frames for accurate HR estimation.

Conclusions:

• The developed framework provides a robust and efficient solution for non-contact HR measurement.
• It demonstrates stability in extracting rPPG signals across diverse and challenging video conditions.
• This approach advances the field of remote physiological monitoring.