CalibrationPhys: Self-Supervised Video-Based Heart and Respiratory Rate Measurements by Calibrating Between Multiple...

Area of Science:

• Biomedical Engineering
• Computer Vision
• Machine Learning

Background:

• Traditional contact-based sensors for physiological monitoring can be intrusive and less user-friendly.
• Current deep learning methods for video-based vital sign estimation often require extensive, costly ground-truth data for training.
• There is a need for efficient and accessible methods for remote physiological monitoring.

Purpose of the Study:

• To develop a self-supervised, video-based method for measuring heart and respiratory rates.
• To eliminate the need for ground-truth physiological data in training deep learning models for vital sign estimation.
• To enable the use of arbitrary cameras for accurate physiological measurements through multi-camera calibration.

Main Methods:

• Proposed CalibrationPhys, a self-supervised learning framework utilizing synchronized facial videos from multiple cameras.
• Employed contrastive learning to train models, treating predictions from synchronized videos as positive pairs and those from different videos as negative pairs.
• Incorporated data augmentation techniques to enhance model robustness and leveraged pre-trained models for camera-specific optimization.

Main Results:

• CalibrationPhys achieved state-of-the-art performance in heart and respiratory rate measurement compared to existing methods.
• The method demonstrated effectiveness across two independent datasets.
• Self-supervised training without ground-truth data yielded highly accurate physiological estimations.

Conclusions:

• CalibrationPhys offers a practical and cost-effective solution for video-based vital sign monitoring.
• The multi-camera calibration approach allows for flexible deployment with various camera setups.
• This method significantly advances the field of remote, non-contact physiological sensing.