Emotional Climate Recognition in Conversations using Peers' Speech-based Bispectral Features and Affect Dynamics

Area of Science:

• Artificial Intelligence
• Human-Computer Interaction
• Affective Computing

Background:

• Emotion recognition in conversations using AI is crucial for understanding social behavior.
• Current methods lack the ability to capture the dynamic, joint emotional atmosphere of conversations.
• Recognizing the emotional climate (EC) is essential for deeper insights into interpersonal dynamics.

Purpose of the Study:

• To propose and evaluate MLBispeC (Machine Learning Based Bispectral Classification), a novel AI-based approach for recognizing the emotional climate (EC) in conversations.
• To extend emotion recognition beyond individual affect to the collective emotional atmosphere.
• To demonstrate the effectiveness of bispectral analysis and machine learning for EC recognition.

Main Methods:

• Utilized time-windowed bispectral analysis on speech signals to extract features from nonlinear harmonic interactions.
• Combined extracted features with affect dynamics from emotion labeling to create an extended feature vector.
• Employed Support Vector Machine and K-Nearest Neighbor classifiers for classification of emotional climate.

Main Results:

• MLBispeC achieved superior performance compared to state-of-the-art methods on the IEMOCAP dataset.
• Reported high accuracy (0.826A/0.754V), sensitivity (0.864A/0.774V), and AUC (0.821A/0.799V) for Arousal and Valence dimensions.
• Demonstrated the objective and dynamic recognition of peers' emotional climate during conversations.

Conclusions:

• MLBispeC effectively recognizes the emotional climate in conversations, providing objective insights into emotional and social interactions.
• The method's ability to dynamically assess EC can aid in the assessment of patients with various physiological, psychological, and mental health conditions.
• This approach offers a valuable tool for unobtrusive, objective, and dynamic assessment across different age groups.