A Real-Time Affective Computing Platform Integrated with AI System-on-Chip Design and Multimodal Signal Processing...

Area of Science:

• Affective computing and artificial intelligence.
• Biomedical signal processing and wearable technology.

Background:

• Deep learning is widely used for emotion recognition but faces real-time processing challenges due to computing power and latency.
• Existing methods struggle with real-time emotion detection, limiting practical applications.

Purpose of the Study:

• To propose a real-time affective computing platform integrating an AI System-on-Chip (SoC) with multimodal physiological signals.
• To enable efficient and accurate emotion recognition for applications like real-time monitoring.

Main Methods:

• Utilized electroencephalogram (EEG), electrocardiogram (ECG), and photoplethysmogram (PPG) signals for multimodal emotion feature extraction.
• Employed short-time Fourier transform (STFT) for EEG and direct signal extraction for ECG/PPG.
• Implemented a Long-term Recurrent Convolutional Network (LRCN) classifier on an AI SoC design, classifying emotions into happy, angry, and sad.

Main Results:

• The LRCN classifier achieved an average cross-subject accuracy of 77.41% for emotion recognition.
• The AI SoC demonstrated efficient processing with on-chip training at 5.5 µs and real-time classification at 1.9 µs per sample.
• The platform provides real-time emotion monitoring via a graphical user interface (GUI) updated every second.

Conclusions:

• The developed platform offers a viable solution for real-time emotion recognition by overcoming computational constraints.
• The integrated AI SoC and multimodal signal processing enable efficient and accurate affective computing.
• This technology has significant clinical relevance for continuous, real-time emotion monitoring.