Related Experiment Video
Updated: May 12, 2026

Examining Gesture Production in the Presence of Communication Challenges
Published on: January 26, 2024
The language of gesture: Testing the multimodality-for-compensation account with head and hand emblems of affirmation
Samuel Sonntag1, Ian Grant Mackenzie1, Emanuel Schütt1
1Department of Psychology, University of Tubingen.
Abstract:
Language is inherently multimodal in nature. However, the extent to which verbal and gestural information are integrated to form a coherent meaning is still not fully understood. Although previous research has focused on iconic gestures, the present study investigates the integration of emblematic gestures-culturally specific gestures with direct verbal translations. Specifically, we examine how basic verbal and gestural affirmative and negative responses to polar questions are processed, both in isolation and in combination. Negation processing is known to be particularly demanding, making it an ideal test case for the multimodality-for-compensation hypothesis, which posits that multimodal cues are especially beneficial under increased processing difficulty. Across three experiments, participants responded to simple yes-no questions answered by either unimodal, compatible, or incompatible multimodal responses ("yes"/"no" in combination with head nods/shakes and thumbs up/down gestures). The results demonstrated consistent integration effects between verbal and gestural information, with stronger integration effects for affirmative compared with negative responses. The implications of these findings for theories of multimodal language processing are discussed. (PsycInfo Database Record (c) 2026 APA, all rights reserved).
Related Concept Videos
Non-Verbal Cues
Channels of Non-Verbal Communication
Therapeutic Communication
Verbal communication depends on language or a prescribed way of using words so that people can share information effectively. The critical aspects of verbal...
Facial Feedback Hypothesis
Language and Cognition
Lateralization
