Improving ROI detection in photoplethysmographic imaging with thermal cameras

Photoplethismographic imaging (PPGi) enables the estimation of heart rate without body contact by analyzing the temporal skin color changes from video recordings. Motion artifacts and atypical facial characteristics cause poor signals and currently limit the applicability of PPGi. We have developed a novel algorithm for locating cheek and forehead region of interests (ROI) with the aim to improve PPGi during challenging situations. The proposed approach is based on the fusion of RGB and far-infrared (FIR) video streams where FIR ROI is used as fall-back when RGB alone fails. We validated and compared the algorithm against the detection based on single sources, using videos from 8 subjects with distinctively different face characteristics. The subject performed three scenarios with incremental motion artifact content (head at rest, intensive head movements, speaking). The results showed that combining the two imaging sources increased the detection rate of cheeks from 75% (RGB) to 92% (RGB+FIR) in the challenging intensive head movement scenario. This work demonstrated that FIR imaging is complementary to simple RGB imaging and when combined, adds robustness to the detection of ROI in PPGi applications.