Exploring the Contributions of Various Acoustic Features in Cantonese Vocal Emotions

The aim of this study was to investigate the acoustic patterns of six emotions and a neutral state in Cantonese speech by focusing on the prosodic modulations that convey emotional content in this tonal language, which has six lexical tones.

We employed the extended Geneva minimalistic acoustic parameter set to systematically analyze the acoustic features of 3,474 recordings from the Cantonese Audio-Visual Emotional Speech Database. Linear mixed-effects models were fitted to examine variations in acoustic parameters across emotional states. Decision tree models were used to assess the relative contributions of 22 acoustic parameters in classifying emotions.

By fitting linear mixed-effects models, our results revealed statistically significant variations in most of the acoustic parameters across diverse emotional states. The decision tree models showed the relative contributions of 22 acoustic parameters in the classification of emotions, with spectral parameters accounting for 65.45% of the significance in distinguishing all seven emotional states, significantly exceeding other groups of features.

Our findings highlight the unique characteristics of emotional expression in Cantonese, in which spectral parameters play a more significant role compared to the frequency-related parameters that are often emphasized in nontonal languages. Our results contribute significantly to understanding vocal emotion expression in tonal languages and are particularly useful for designing emotion-recognition systems and hearing aids that are tailored to tonal language environments. Furthermore, these insights have potential implications for enhancing emotional communication and cognitive training interventions for Cantonese-speaking individuals who use hearing aids or have cochlear implants, are on the autism spectrum, or have Alzheimer's disease.