Linguistic-visual routines and name activation in visual world studies of spoken word recognition

We address a longstanding issue in visual world studies of speech and spoken word recognition: What role does retrieval of object names play in mediating eye movements to displayed objects? We propose a novel "linguistic-visual routines" linking hypothesis in which the representations of spoken words are linked to schematic visual representations and visuomotor routines, without involvement of object names. When triggered during reference resolution, these routines direct gaze to objects in a co-present visual world that are consistent with the schematic visual representations. To evaluate possible picture-naming in this task, we created materials that included "synonym" objects (e.g., a picture of an object that could be named as either "couch" or "sofa", where norms established that each name was an equally good fit for the object, but one name was dominant, in this case, "couch". Name typicality effects were used to diagnose if and when object names affect eye movements, using targets that were cohorts of either the dominant or subordinate name of a competitor. We reasoned that if names were retrieved, then there would be larger cohort effects and delayed looks to targets when the target was a cohort of the dominant name. We ran conceptual simulations using normalized recurrence localist attractor networks, illustrating predicted data patterns for models where: (1) object names do not affect fixations; (2) retrieval of object names mediates fixations, and (3) sub-threshold priming of object names weakly affects eye-movements. Experiment 1 used a standard Visual World design, finding minimal evidence, if any, for effects of name typicality. In Experiments 2 and 3, participants were cued that a particular object would be occluded before the instruction begins and thus were likely to retrieve the object name to help maintain it in memory. On critical trials, synonym objects were occluded in Experiment 2, and distractor objects were occluded in Experiment 3. Simulations were again used to illustrate predicted data patterns, namely name-retrieval effects only when the synonym object was occluded. In Experiment 2, where the synonym object was occluded, there were strong name typicality effects, with larger cohort effects and delayed target looks for the dominant name, matching the predictions of the model with name retrieval. In Experiment 3, where a distractor object was occluded, cohort effects were not affected by name typicality. We then introduce models simulating conditions from a pilot study in which visually drawing attention to an object, but not occluding it, also does not result in name typicality effects. Taken together, the results are clearly consistent with the linguistic-visual routines hypothesis and inconsistent with name retrieval. We conclude that object name representations play at most a minimal role in mediating fixations in Visual World experiments of spoken word recognition.