An effective multi-modality analysis for stress classification: A signal-to-image conversion using local pattern...

Stress is an intensified reaction that occurs when humans experience challenges(stressors) due to complex and nonlinear responses. The study proposes a pattern-driven framework that combines signal and image-based modalities, incorporating Local Binary Pattern (LBP), Local Normal Derivative Pattern (LNDP), Local Derivative Pattern (LDP), and Local Tetra Pattern (LTrP) using Spectrogram. Derived from the analyzed patterns, features spanning in the Time Domain, Non-Linear chaos theory, Fractal Dimensions, and Histogram descriptors are extracted. This study employs binary classification techniques to identify stress by distinguishing between baseline and stress states in multi-modality sensor data, such as heart rate (HR), and respiration rate (RR). In signal analysis, statistical and non-linear features related to the predictability of stress attained the high classification accuracy using a Support Vector Machine (SVM) based on its linearity. Logistic Regression (LG) with its data complexity has obtained a good accuracy for overall features. For image analysis, the LBP technique exhibits strong overall classification performance for statistical at 98.7 %, entropy and histogram at 100 %, and fractals at 90 % compared with methods. Specifically, for LNDP and LTrP, Logistic Regression and Ensemble outperform SVM, achieving an impressive accuracy of 100 %, and superior performance metrics such as Precision, Matthews Correlation Coefficient (MCC), and Kappa Score based on the distribution of pixel intensities and directionality of pattern images as 0.4 to 1. The AlexNet gives good classification accuracy for Image Analysis of stress detection. Based on the intricate patterns at different scales, image analysis of spectrograms and pattern techniques yields better classification accuracy compared with signal analysis for stress prediction.