Visual arguments

Three experiments investigated the use of verb argument structure by tracking participants' eye movements across a set of related pictures as they listened to sentences. The assumption was that listeners would naturally look at relevant pictures as they were mentioned or implied. The primary hypothesis was that a verb would implicitly introduce relevant entities (linguistic arguments) that had not yet been mentioned, and thus a picture corresponding to such an entity would draw anticipatory looks. For example, upon hearing ...mother suggested..., participants would look at a potential recipient of the suggestion. The only explicit task was responding to comprehension questions. Experiments 1 and 2 manipulated both the argument structure of the verb and the typicality/co-occurrence frequency of the target argument/adjunct, in order to distinguish between anticipatory looks to arguments specifically and anticipatory looks to pictures that were strongly associated with the verb, but did not have the linguistic status of argument. Experiment 3 manipulated argument status alone. In Experiments 1 and 3, there were more anticipatory looks to potential arguments than to potential adjuncts, beginning about 500 ms after the acoustic onset of the verb. Experiment 2 revealed a main effect of typicality. These findings indicate that both real world knowledge and argument structure guide visual attention within this paradigm, but that argument structure has a privileged status in focusing listener attention on relevant aspects of a visual scene.