MFF-Net: A Lightweight Multi-Frequency Network for Measuring Heart Rhythm from Facial Videos.

Area of Science:

• Biomedical Engineering
• Signal Processing
• Artificial Intelligence

Background:

• Remote photoplethysmography (rPPG) offers camera-based health monitoring by analyzing facial videos for heart rhythm.
• Existing deep learning models for rPPG achieve high accuracy but struggle with illumination variations, motion artifacts, and computational demands.
• These limitations hinder the practical application of rPPG in real-world scenarios.

Purpose of the Study:

• To develop a lightweight and efficient deep learning network, MFF-Net, for accurate heart rhythm measurement from facial videos.
• To address the challenges of illumination variation, motion artifacts, and computational burden in rPPG analysis.
• To enable robust and accessible real-time health monitoring using facial video data.

Main Methods:

• Proposed a multi-frequency mode signal fusion (MFF) mechanism to separate and process different modes of rPPG signals, enhancing accuracy in noisy environments.
• Introduced a temporal multi-scale convolution module (TMSC-module) for expanded receptive fields and richer multi-scale information extraction.
• Developed a spectrum self-attention module (SSA-module) to aid in signal reconstruction and multi-dimensional signal merging.
• Implemented an over-fitting sampling training scheme to improve network generalization and fitting ability.

Main Results:

• MFF-Net demonstrated superior performance in estimating heart rate (HR) and heart rate variability (HRV) compared to state-of-the-art methods on benchmark datasets.
• The proposed network achieved these results with a significantly lower computational burden.
• The MFF mechanism, TMSC-module, and SSA-module effectively improved the recovery of blood volume pulse (BVP) signals under complex noise conditions.

Conclusions:

• MFF-Net presents a computationally efficient and robust solution for rPPG-based heart rhythm monitoring.
• The developed network effectively mitigates common challenges in rPPG analysis, such as illumination variations and motion artifacts.
• MFF-Net shows significant potential for deployment in various real-world health monitoring applications.