Communicative efficiency in multimodal language directed at children and adults

Area of Science:

• Psycholinguistics
• Cognitive Science
• Communication Studies

Background:

• Human communication relies on multimodal adjustments in vocal and gestural channels.
• Speakers dynamically modify speech rate and gestures during face-to-face interaction.
• The functional role of these dynamic adjustments in communicative success and cognitive effort remains under investigation.

Purpose of the Study:

• To investigate whether speakers dynamically adjust vocal and gestural communication to enhance communicative success and decrease cognitive effort.
• To examine if word predictability, operationalized as surprisal, influences vocal (word duration) and gestural (iconic gestures) adjustments.
• To determine if these effects differ between adult-child and adult-adult communication.

Main Methods:

• Analysis of a novel corpus (Ecological Language Corpus) of naturalistic adult-child (3-4 years) and adult-adult interactions.
• Computation of word surprisal using computational language models trained on child-directed and adult-directed corpora.
• Assessment of correlations between word surprisal and modulations in word duration and iconic gesture production.

Main Results:

• Word surprisal significantly predicts speakers' multimodal adjustments in both vocal and gestural channels.
• Some adjustments were modulated by the comprehender's age (child vs. adult).
• Communicative efficiency extends across vocal and gestural modalities, not limited to language structure or vocal modality alone.

Conclusions:

• Speakers dynamically adjust multimodal communication based on word predictability to optimize communicative efficiency.
• These findings highlight the general applicability of communicative efficiency principles across vocal and gestural channels.
• The study provides evidence for the role of surprisal in modulating both spoken and gestural aspects of communication.