Multi-resolution modulation-filtered cochleagram feature for LSTM-based dimensional emotion recognition from speech.

Area of Science:

• Speech emotion recognition
• Human-robot interaction
• Auditory perception modeling

Background:

• Continuous dimensional emotion recognition from speech is crucial for natural human-robot interactions.
• Temporal dynamics of emotional states are better captured by time-domain auditory perception cues than frequency-domain acoustic features.
• Extracting temporal dynamics of emotion from speech is challenging due to complex auditory models.

Purpose of the Study:

• To investigate multi-resolution representations of auditory perception for emotion recognition.
• To propose a novel feature, multi-resolution modulation-filtered cochleagram (MMCG), for predicting valence and arousal.
• To develop a parallel long short-term memory (LSTM) architecture to model multi-temporal dependencies in MMCG features.

Main Methods:

• Constructed MMCG by combining four modulation-filtered cochleagrams at different resolutions.
• Designed a parallel LSTM architecture to process MMCG features and capture multi-temporal dependencies.
• Conducted experiments on the RECOLA and SEWA datasets to evaluate feature performance.

Main Results:

• MMCG demonstrated superior recognition performance compared to all other evaluated features on both datasets.
• The parallel LSTM architecture effectively modeled multi-temporal dependencies from MMCG.
• The proposed method achieved better valence and arousal prediction performance than a standard LSTM.

Conclusions:

• MMCG is an effective feature for capturing temporal dynamics in speech emotion recognition.
• The parallel LSTM architecture enhances emotion prediction by modeling multi-temporal dependencies.
• This approach advances the capabilities of robots and virtual agents in understanding human emotional states.