Sometimes you look more when you know less: Goal dependent eye movements during language processing

In the literature on language comprehension during concurrent scene viewing (i.e., the "visual world paradigm"), many studies tacitly assume that the more certain listeners are that an entity is being referred to, the more they gaze at it-and findings often appear to support this assumption. Here we directly test this assumption across two experiments that used identical linguistic materials and visual scenes but differed in task demands. In a task in which participants were instructed to act upon the referent of a pronoun, we observed the typical positive relationship between referential certainty and visual attention to the entity most likely to be the pronoun's referent. However, in a comprehension-only task (i.e., "look-and-listen"), greater referential certainty did not increase looks to the likely referent. Instead, referential certainty was positively related to looks to entities that were not potential referents. Thus, the relationship between referential certainty and visual attention depended on task context. These findings show that eye movements in the visual world paradigm do not necessarily shift to entities whose representations are currently active. Instead, the goals of the listener shape how visual attention is allocated during comprehension. Thus, it is important to consider task context when interpreting eye movements in the visual world paradigm. We speculate that the pattern observed in the comprehension-only task may arise because rather than focusing visual attention on entities that are already well-established in the discourse, listeners may direct their visual attention toward entities that they expect will be more informative given their goal of comprehending the language. We relate our findings to similar phenomena observed during reading, scene description and viewing, and in infants' gaze preferences, as well as to related effects in language production.