Remote Heart Rate Measurement from RGB-NIR Video Based on Spatial and Spectral Face Patch Selection

Area of Science:

• Biomedical Engineering
• Computer Vision
• Medical Imaging

Background:

• Accurate heart rate (HR) estimation is crucial for remote patient monitoring.
• Existing methods using only RGB cameras struggle in low-light or fluctuating light conditions.
• There is a need for robust HR estimation techniques that perform well across diverse environmental scenarios.

Purpose of the Study:

• To develop a novel heart rate estimation method utilizing simultaneously recorded RGB and near-infrared (NIR) face videos.
• To enhance the robustness of HR estimation by automatically selecting optimal face patches in both spatial and spectral domains.
• To demonstrate superior performance compared to existing RGB-only methods, particularly in challenging lighting environments.

Main Methods:

• Simultaneous recording of face videos using both RGB and NIR cameras.
• Development of an algorithm for automatic selection of salient face patches for HR estimation.
• Application of spatial and spectral domain analysis for robust feature extraction.
• Validation of the proposed method on subjects under various lighting conditions, including low light and light fluctuations.

Main Results:

• The proposed method achieved successful HR estimation for all 20 subjects under challenging low-light conditions (within ±3 beats per minute).
• In contrast, RGB camera-based methods only succeeded for 25% of subjects in the same challenging scenarios.
• The spatial and spectral face patch selection demonstrated significant improvements in robustness across different environments.

Conclusions:

• The novel HR estimation method combining RGB and NIR video data offers superior robustness and accuracy, especially in adverse lighting.
• Automatic spatial and spectral face patch selection is key to overcoming limitations of existing camera-based HR monitoring techniques.
• This approach holds promise for reliable remote photoplethysmography (rPPG) in real-world, uncontrolled settings.