Emotional Speech Recognition Method Based on Word Transcription

Area of Science:

• Computer Science
• Artificial Intelligence
• Linguistics

Background:

• The COVID-19 pandemic necessitated remote learning, increasing the need for tools to monitor student engagement and emotional states during online assessments.
• Traditional speech recognition systems often lack the capability to interpret emotional nuances within spoken language.

Purpose of the Study:

• To develop and evaluate an emotional speech recognition method for identifying student emotions during online exams.
• To assess the accuracy and applicability of the proposed method across different languages and contexts.

Main Methods:

• A novel emotional speech recognition approach utilizing a knowledge base of emotionally charged words (codebook).
• Speech signal processing involving signal capture, speech detection, simplified transcription, word boundary determination, and comparison with the codebook.
• Experimental validation using 420 audio recordings to assess method performance.

Main Results:

• The method achieved an accuracy of 79.7% for the Kazakh language in recognizing emotions within spoken speech.
• Demonstrated effectiveness in identifying positive and negative emotions, with complete word recognition when emotions are present.
• The method is computationally inexpensive, allowing for widespread application.

Conclusions:

• The developed emotional speech recognition method is effective and accurate, particularly for the Kazakh language.
• The technique's low computational demands facilitate its broad deployment in diverse environments like schools and public spaces.
• Future applications include developing systems for real-time threat detection based on emotional speech analysis.