Hierarchical prosody modeling for Mandarin spontaneous speech.

Area of Science:

• Computational Linguistics
• Speech Processing
• Phonetics

Background:

• Prosody plays a crucial role in speech understanding and recognition.
• Existing models often struggle to capture the complex hierarchical nature of prosody in spontaneous speech.
• Mandarin spontaneous speech presents unique prosodic challenges due to its tonal nature and disfluencies.

Purpose of the Study:

• To propose a novel hierarchical prosody model (HPM) for Mandarin spontaneous speech.
• To investigate the relationship between acoustic features, linguistic information, and hierarchical prosodic structures.
• To apply the developed HPM to enhance Mandarin spontaneous speech recognition.

Main Methods:

• Designed a hierarchical prosody model (HPM) to integrate acoustic, linguistic, and prosodic tag features.
• Employed a sequential optimization algorithm for unsupervised training of the HPM using the Mandarin Conversational Dialogue Corpus (MCDC).
• Automatically labeled utterances with break and prosodic state tags to represent hierarchical structures.

Main Results:

• Identified critical prosodic properties of Mandarin spontaneous speech by examining HPM parameters.
• Mapped prosodic events onto hierarchical structures, enabling detailed prosodic analyses, including disfluencies.
• Achieved significant relative error rate reductions: 9.0% (base-syllable), 9.2% (character), 15.6% (tone), and 7.3% (word) in speech recognition.

Conclusions:

• The proposed HPM effectively models hierarchical prosody in Mandarin spontaneous speech.
• The HPM facilitates a deeper understanding of Mandarin prosodic properties and disfluencies.
• The HPM demonstrates practical utility in improving the accuracy of Mandarin spontaneous speech recognition systems.