Language prediction in multimodal contexts: The contribution of iconic gestures to anticipatory sentence...

Evidence shows listeners predict upcoming words, and iconic gestures in face-to-face dialogue often precede their spoken referents. A recent framework proposes that early-onset gestures can pre-activate the meaning of upcoming nouns. This pre-registered EEG study tested this proposal. 81 Dutch adults watched videos of an actress producing discourses ending with a target noun. The noun was either predictable or not from the discourse. Each noun was paired with an iconic gesture whose stroke began either 520 msec (earlier) or 130 msec (later) before the noun. Discourse predictability produced the hypothesized post-target N400 reduction for predictable versus non-predictable items (H1). In the pre-target window, earlier gestures in non-predictable discourses elicited more negative amplitudes than in predictable discourses (H2a), suggesting that earlier gestures modulated neural responses when predictability was low. No comparable interaction was found for the opposite contrast (early-predictable vs late-predictable & late-non-predictable; H2c). In the post-target window, earlier gestures yielded a modest N400-like positivity relative to late gestures (H2e), but the interaction between gesture timing and discourse predictability (H2f) was not supported. A follow-up indicated that, in non-predictable discourses, later gestures produced a larger N400 than earlier gestures. Thus, the data confirm H1 and support H2a (and partially H2e), while H2c and H2f were not supported, suggesting that when speakers produce monologic discourses, earlier iconic gestures can aid anticipatory processing when linguistic predictions are weak, but their effect is limited and context-dependent. Together with the exploratory follow-up, the findings highlight a nuanced role of the influence of gesture timing on comprehension.