Leveraging Linguistic Context in Dyadic Interactions to Improve Automatic Speech Recognition for Children

Area of Science:

• Speech Technology
• Computational Linguistics
• Developmental Psychology

Background:

• Automatic speech recognition (ASR) for child speech is challenging due to biological changes, developing language skills, and limited data.
• Spontaneous child speech in conversational interactions, particularly with communication impairments, presents further difficulties for ASR systems.
• Health applications, such as behavioral assessments using dyadic interactions, motivate the need for robust child ASR.

Purpose of the Study:

• To adapt ASR models for child speech using linguistic context from adult interlocutors.
• To investigate methods for exploiting adult speech context, including lexical repetitions and semantic response generation.
• To evaluate the effectiveness of context-adapted models in improving ASR accuracy for children across different interaction types.

Main Methods:

• Utilized sequence-to-sequence models to predict child utterances based on adult speech context.
• Incorporated long-term conversational context by propagating cell-state across interactions.
• Employed utterance-level language model adaptation using interpolation, analyzing context length and direction (forward/backward).

Main Results:

• Context-adapted ASR models demonstrated significant improvements, reducing word error rate by up to 10.71% compared to baseline models.
• The proposed methods showed consistent performance across various context window sizes and directions.
• Statistical analysis confirmed the impact of both adult (source) and child (target) factors on adaptation effectiveness.

Conclusions:

• Leveraging adult speech context effectively improves ASR accuracy for child speech.
• The developed approach offers a generalized solution applicable to diverse child-adult interaction scenarios.
• This work highlights the potential of information transfer from adult interlocutors to enhance child-directed ASR systems.