Multicenter Evaluation of Machine-Learning Continuous Pulse Rate Algorithm on Wrist-Worn Device

Though wrist-worn photoplethysmography (PPG) sensors play an important role in long-term and continuous heart rhythm monitoring, signals measured at the wrist are contaminated by more intense motion artifacts compared to other body locations. Machine learning (ML)-based algorithms can improve long-term pulse rate (PR) tracking but are associated with more stringent regulatory requirements when intended for clinical use. This study aimed to evaluate the accuracy of a digital health technology using wrist-worn PPG sensors and an ML-based algorithm to measure PR continuously.

Volunteers were enrolled in three independent clinical trials and concurrently monitored with the investigational device and FDA-cleared electrocardiography (ECG) devices during supervised protocols representative of real-life activities. The primary acceptance threshold was an accuracy root-mean-square (ARMS) ≤3 beats per minute (bpm) or 5 bpm under no-motion and motion conditions, respectively. Bias, mean absolute error (MAE), mean absolute percentage error (MAPE), limits of agreement (LoA), and Pearson and Lin's concordance correlation coefficients (⍴ and CCC) were also computed. Subgroup and outlier analyses were conducted to examine the effect of site, skin tone, age, sex, body mass index (BMI), and health status on PR accuracy.

Collectively, 16,915 paired observations between the device and the reference ECG were analyzed from 157 subjects (male: 49.04%, age mean: 43 years, age range: 19-83 years, BMI mean: 26.4, BMI range: 17.5-52, Fitzpatrick class V-IV: 22.9%, cardiovascular condition: 24%). The PR output attained an accuracy of 1.67 bpm under no-motion (n = 5,621 min) and 4.39 bpm under motion (n = 11,294 min), satisfying the acceptance thresholds. Bias and LoA (lower, upper LoA) were -0.09 (-3.36, 3.17) bpm under no-motion and 0.51 (-8.05, 9.06) bpm under motion. MAE was 0.6 bpm in no-motion and 1.77 bpm in motion, and MAPE was 0.86% in no-motion and 2.05% in motion, with ⍴ and CCC >0.98 in both conditions. ARMS values met the clinical acceptance threshold in all relevant subgroups at each clinical site separately, excluding male subjects under motion conditions (ARMS = 5.41 bpm), with more frequent and larger outliers due to stronger forearm contractions. However, these mostly occurred in isolation and, therefore would not impact the clinical utility or usability of the device for its intended use of retrospective review and trend analysis (⍴ and CCC >0.97 and MAPE = 2.61%).

The analytical validation conducted in this study demonstrated clinical-grade accuracy and generalizability of ML-based continuous PR estimations across a full range of physical motions, health conditions, and demographic variables known to confound PPG signals, paving the way for device usage by populations most likely to benefit from continuous PR monitoring.