Using Hybrid HMM/DNN Embedding Extractor Models in Computational Paralinguistic Tasks

Area of Science:

• Computational paralinguistics
• Speech processing
• Machine learning

Background:

• Computational paralinguistics analyzes non-verbal speech content for tasks like emotion recognition.
• Key challenges include handling variable utterance lengths and limited training data.
• Existing methods like x-vectors have limitations in these areas.

Purpose of the Study:

• To present a novel feature extraction method for computational paralinguistics.
• To address limitations of existing methods in handling varying utterance lengths and small corpora.
• To improve performance across diverse paralinguistic tasks.

Main Methods:

• A hybrid Hidden Markov Model/Deep Neural Network (HMM/DNN) acoustic model was trained on an Automatic Speech Recognition (ASR) corpus.
• Embeddings from the ASR model were used as features for paralinguistic tasks.
• Five aggregation methods (mean, std dev, skewness, kurtosis, non-zero ratio) were explored to derive utterance-level features.

Main Results:

• The proposed feature extraction method consistently outperformed the baseline x-vector method.
• Performance gains were observed across various paralinguistic tasks.
• Combining aggregation techniques further enhanced results, depending on the task and neural network layer.

Conclusions:

• The presented method offers a competitive and resource-efficient approach for computational paralinguistics.
• It effectively handles varying utterance lengths and small corpora.
• The technique shows promise for applications in remote monitoring and speech analysis.