Gesture-Speech Interaction Beyond Planning: Evidence from Perturbations During Iconic Gesture and Speech Execution

Area of Science:

• Cognitive Science
• Psycholinguistics
• Neuroscience

Background:

• Gesture and speech are closely coordinated during human communication.
• The precise timing and mechanisms of this coordination, particularly during execution, are debated.
• Competing theories include the interactive view (continuous influence) and the ballistic view (pre-execution limitation).

Purpose of the Study:

• To investigate the relationship between gesture and speech execution.
• To test whether disrupting one modality affects the execution of the other.
• To provide evidence for either the interactive or ballistic view of communication.

Main Methods:

• Four experiments were conducted involving participants naming shapes and depicting events.
• Disruption techniques included virtual reality, motion tracking, and altered sensory feedback (visual/auditory).
• Execution times for both speech and gesture were measured under disrupted conditions.

Main Results:

• Disrupting either gesture or speech consistently increased the execution time of the other modality.
• The degree of execution prolongation in one modality correlated with the prolongation in the other.
• No significant difference was found in the disruption effect based on event type (shapes vs. motion).

Conclusions:

• Gesture and speech remain continuously and bidirectionally linked during the execution phase.
• The findings strongly support the interactive view of communication.
• This suggests ongoing mutual influence between modalities throughout production.