Comparing stress prediction models using smartwatch physiological signals and participant self-reports

Recent advances in wearable technology have facilitated the non-obtrusive monitoring of physiological signals, creating opportunities to monitor and predict stress. Researchers have utilized machine learning methods using these physiological signals to develop stress prediction models. Many of these prediction models have utilized objective stressor tasks (e.g., a public speaking task or solving math problems). Alternatively, the subjective user responses with self-reports have also been used for measuring stress. In this paper, we describe a methodological approach (a) to compare the prediction performance of models developed using objective markers of stress using participant-reported subjective markers of stress from self-reports; and (b) to develop personalized stress models by accounting for inter-individual differences. Towards this end, we conducted a laboratory-based study with 32 healthy volunteers. Participants completed a series of stressor tasks-social, cognitive and physical-wearing an instrumented commercial smartwatch that collected physiological signals and participant responses using timed self-reports. After extensive data preprocessing using a combination of signal processing techniques, we developed two types of models: objective stress models using the stressor tasks as labels; and subjective stress models using participant responses to each task as the label for that stress task. We trained and tested several machine learning algorithms-support vector machine (SVM), random forest (RF), gradient boosted trees (GBT), AdaBoost, and Logistic Regression (LR)-and evaluated their performance. SVM had the best performance for the models using the objective stressor (i.e., stressor tasks) with an AUROC of 0.790 and an F-1 score of 0.623. SVM also had the highest performance for the models using the subjective stress (i.e., participant self-reports) with an AUROC of 0.719 and an F-1 score of 0.520. Model performance improved with a personalized threshold model to an AUROC of 0.751 and an F-1 score of 0.599. The performance of the stress models using an instrumented commercial smartwatch was comparable to similar models from other state-of-the-art laboratory-based studies. However, the subjective stress models had a lower performance, indicating the need for further research on the use of self-reports for stress-related studies. The improvement in performance with the personalized threshold-based models provide new directions for building stress prediction models.