Contactless blood oxygen estimation from face videos: A multi-model fusion method based on deep learning

Area of Science:

• Biomedical Engineering
• Computer Vision
• Physiological Monitoring

Background:

• Blood oxygen saturation ($$) is a critical indicator of respiratory health, gaining importance during the COVID-19 pandemic.
• Lower blood oxygen levels are observed in COVID-19 patients prior to symptom manifestation.
• Existing methods for monitoring $$ via facial videos have limitations.

Purpose of the Study:

• To develop a novel deep learning-based multi-model fusion method for accurate blood oxygen saturation ($$) estimation from facial videos.
• To address the shortcomings of current video-based $$ monitoring techniques.
• To facilitate remote medicine and home health applications through improved $$ estimation.

Main Methods:

• A feature extraction network, Residuals and Coordinate Attention (RCA), was employed, utilizing residual blocks and coordinate attention for enhanced feature correlation and spatial information.
• A multi-model fusion module integrated the Color Channel Model (CCM) and Network-Based Model (NBM).
• The CCM used an image generator to reconstruct color channel signals for $$ calculation, while the NBM incorporated a novel two-part loss function to minimize interference from other physiological signals.

Main Results:

• The proposed multi-model fusion method (MMFM) achieved a mean absolute error below 2%, meeting clinical requirements.
• Experimental results on the PURE and VIPL-HR datasets validated the effectiveness of the three integrated models.
• The fusion approach successfully leveraged facial video features to enhance $$ estimation performance.

Conclusions:

• The developed multi-model fusion deep learning method provides accurate blood oxygen saturation ($$) estimation from facial videos.
• This technique effectively utilizes color features and mitigates signal disturbances, outperforming existing methods.
• The research supports advancements in remote patient monitoring and telehealth solutions.