Heart rate estimation network from facial videos using spatiotemporal feature image

Area of Science:

• Biomedical Engineering
• Signal Processing
• Artificial Intelligence

Background:

• Remote health monitoring is increasingly vital post-pandemic.
• Contactless vital sign measurement, like Heart Rate (HR), is challenging due to weak physiological signals and noise.
• Noise sources include head movements, illumination variations, and acquisition device limitations.

Purpose of the Study:

• To propose a video-based, noise-less method for cardiopulmonary measurement.
• To enable accurate Heart Rate (HR) estimation from remote photoplethysmography (rPPG) signals.
• To overcome challenges of heterogeneous lighting and continuous motion in remote sensing.

Main Methods:

• Converting 3D videos to 2D Spatio-Temporal Images (STI) to suppress noise and preserve temporal rPPG signal information.
• Developing a novel motion representation using wavelets for Convolutional Neural Networks (CNN).
• Forming STI by concatenating wavelet-decomposed feature vectors from sequential frames, feeding into CNN for HR estimation.

Main Results:

• The proposed approach effectively suppresses noise while retaining crucial temporal information.
• The model demonstrates accurate HR estimation under challenging conditions, including varying illumination and motion.
• Superior performance was validated on four benchmark datasets: MAHNOB-HCI, MMSE-HR, UBFC-rPPG, and VIPL-HR.

Conclusions:

• The developed video-based method offers a robust solution for noise-less remote Heart Rate (HR) monitoring.
• The use of Spatio-Temporal Images (STI) and wavelet-derived motion representation enhances CNN's pattern visualization capabilities for physiological signals.
• This approach holds significant potential for advancing remote healthcare and continuous patient monitoring.