A GPTAssisted Multi Modal Emotion Intelligence Framework for Mental Health Predictive Analytics using Physiological...

Area of Science:

• Biomedical Engineering
• Artificial Intelligence in Healthcare
• Affective Computing

Background:

• Improving predictive analytics in healthcare requires robust frameworks for interpreting complex physiological signals.
• Current methods often struggle with multi-modal data integration and nuanced emotional state recognition.

Purpose of the Study:

• To develop and evaluate a multi-modal emotion recognition architecture for mental health care.
• To integrate electroencephalogram (EEG), electrocardiogram (ECG), and galvanic skin (GS) recordings with a GPT-based natural language processing (NLP) interface.
• To enhance clinical reasoning and healthcare decision-making through improved interpretability.

Main Methods:

• Sophisticated preprocessing and synchronization using cross-correlation and discrete wavelet transform for noise reduction.
• Feature extraction via wavelet scattering transform and statistical methods, followed by dimensionality reduction using 2D-bidirectional principal component collaborative projection.
• Feature fusion using canonical correlation analysis and classification using an optimized Whale Optimization Algorithm-based Kernel Extreme Learning Machine (WOA-KELM) model.
• Integration of a GPT module for post-classification interpretation of clinical text and self-reported emotions.

Main Results:

• The WOA-KELM model significantly outperformed traditional classifiers (SVM, k-NN, XGBoost) and standard KELM.
• High classification rates achieved for valence (96.93%) and arousal (99.05%) in binary emotion recognition tasks.
• GPT-based contextual embeddings improved interpretability scores, aiding clinical reasoning and healthcare decision-making.

Conclusions:

• The proposed multi-modal framework offers a credible solution for emotion-aware diagnostics and mental health monitoring.
• Optimized multi-model solutions, including GPT integration, can meaningfully facilitate predictive healthcare analytics.
• This approach paves the way for real-time, personalized healthcare solutions and adaptive human-computer interaction.