Linear and nonlinear analysis of multimodal physiological data for affective arousal recognition

In this work we intend to design a system to classify human arousal at five levels (i.e., five stress levels) using four peripheral bio signals including photo-plethysmography measurements (PPG), galvanic skin response (GSR), thorax respiration (TR) and abdominal respiration (AR).

A total of 98 young people voluntarily participated in this study, including 65 men and 33 women with an average age of 24.48 ± 4.26 years. We induced five levels of mental stress in subjects through the Stroop test. A range of physiological features from different analysis domains, including statistical, frequency, and geometrical analyzes, as well as recurrence quantification analysis (RQA) and detrended fluctuation analysis (DFA) were extracted to find out the best arousal-related features and to correlate them with arousal states. Classification of the five arousal levels is performed by a simple naïve Bayes classifier.

Accuracies of 58.45%, 57.1% and 69.13% were obtained using linear features, nonlinear features and a combination of them, respectively. The combination of linear and nonlinear features resulted in the largest average accuracy of 69.13%, ICC of 88.12% and F1 of 69.43% values in the classification of five levels of mental stress.

This paper suggested that combining linear and nonlinear dynamic methods for the analysis of physiological data could help improve the accuracy of the recognition of arousal levels. However, it should be noted that combining multiple modalities (here, PPG, GSR and respiration modalities) by equally weighting them may not always be a good approach to improve accuracy.