Speech emotion recognition with light weight deep neural ensemble model using hand crafted features.

Area of Science:

• Artificial Intelligence
• Human-Computer Interaction
• Signal Processing

Background:

• Speech Emotion Recognition (SER) is vital for healthcare, HCI, and robotics.
• Challenges in SER include data scarcity and complex feature extraction.
• Existing methods often rely on automatic feature extraction like spectrograms.

Purpose of the Study:

• To investigate if a lightweight deep neural ensemble model (CNN & CNN_Bi-LSTM) with hand-crafted features surpasses automatic feature extraction methods for SER.
• To evaluate the impact of fine-tuning techniques like learning rate schedulers and regularization on model performance.
• To validate the proposed model on diverse, publicly available SER datasets.

Main Methods:

• Developed a CNN and CNN_Bi-LSTM ensemble model.
• Utilized hand-crafted audio features: Zero Crossing Rate (ZCR), Root Mean Square Error (RMSE), Chroma Short-Time Fourier Transform (STFT), and Mel-Frequency Cepstral Coefficients (MFCC).
• Employed learning rate schedulers, regularization, and the LIME technique for model interpretation across five datasets (RAVDESS, TESS, SAVEE, CREMA-D, EmoDB).

Main Results:

• The proposed ensemble model consistently outperformed individual models and spectrogram-based approaches.
• Achieved superior performance across key metrics: Accuracy, AUC-ROC, AUC-PRC, and F1-score.
• LIME analysis provided interpretability for the model's predictions.

Conclusions:

• Lightweight ensemble models with carefully selected hand-crafted features are highly effective for SER.
• Fine-tuning strategies are crucial for optimizing deep learning models in emotion recognition.
• The proposed approach offers a robust and interpretable solution for automatic emotion detection from speech.