Imposing Cognitive Constraints on Reference Production: The Interplay Between Speech and Gesture During Grounding

Area of Science:

• Psycholinguistics
• Cognitive Science
• Communication Studies

Background:

• Establishing common ground in conversation is crucial for effective communication.
• The role of visual cues, specifically co-speech gestures, in multimodal grounding is underexplored.
• The impact of cognitive constraints on how speakers integrate verbal and gestural information during grounding needs further investigation.

Purpose of the Study:

• To investigate the interplay between verbal and gestural modalities during grounding in referential communication.
• To examine how repetition and cognitive load influence multimodal grounding.
• To understand the coordination and independent unfolding of speech and gesture in language production.

Main Methods:

• A collaborative referential communication task was designed.
• Participants engaged in repeated references to objects or concepts.
• A time constraint was imposed to manipulate cognitive load.

Main Results:

• No significant differences were found in the semantic-based gesture rate due to repetition or cognitive load.
• Representational gestures and speech appear closely coordinated during the grounding process.
• Differences in the execution of speech and gestures emerged, particularly under time pressure.

Conclusions:

• Speech and gesture function as complementary streams in language production.
• While potentially planned together, speech and gesture production can unfold independently.
• Speakers may prioritize the visual form of gestures over verbal content to optimize collaborative task outcomes under cognitive load.