Remote Heart Rate Prediction in Virtual Reality Head-Mounted Displays Using Machine Learning Techniques.

Area of Science:

• Biomedical Engineering
• Computer Vision
• Physiological Monitoring

Background:

• Head-mounted displays (HMDs) offer potential for remote physiological monitoring via integrated sensors.
• Heart rate is a critical vital sign for assessing user well-being and activity.
• Existing methods lack the ability to predict heart rate solely from facial regions captured by HMDs.

Purpose of the Study:

• To remotely estimate heart rate from facial regions using HMDs.
• To adapt state-of-the-art techniques (EVM-CNN, Meta-rPPG) for HMD-based remote heart rate monitoring.
• To develop a simulated dataset specific to HMD-captured facial regions (eyes, lower face).

Main Methods:

• Developed a region of interest extractor using stabilizer and video magnification for dataset simulation.
• Combined Support Vector Machine (SVM) and FaceMash for region identification.
• Adapted photoplethysmography (PPG) and beats per minute (BPM) prediction algorithms for HMD data.
• Utilized EVM-CNN and Meta-rPPG techniques for heart rate estimation.

Main Results:

• Achieved significant accuracy improvements: 188.88% for EVM and 55.93% for Meta-rPPG.
• Demonstrated successful heart rate prediction using only facial regions as input.
• The adapted Meta-rPPG technique surpassed original performance; EVM adaptation yielded comparable PPG signal results.

Conclusions:

• Remote heart rate monitoring using HMDs and facial analysis is feasible.
• The developed simulation mechanism and adapted algorithms enable accurate, non-invasive heart rate estimation.
• This approach holds promise for enhanced user monitoring in various virtual reality and remote healthcare applications.