Quantifying the Relationship Between Infants' Haptic and Visual Response to Word-Object Pairings

The use of visual and haptic measures as proxies for underlying cognitive abilities has a rich history in infant development research, and perhaps none more so than studies of early vocabulary comprehension. Although visual and haptic measures have displayed long-term predictive value, it remains unclear whether visual attention and haptic responses are analogous and substitutable measures of lexical knowledge. The goal of the current study was to assess the bidirectional relationship between vision and action, and to evaluate the underlying word representations that guide infants' visual and haptic responses. Two measures of visual attention, one macro-level (look accuracy) and one micro-level (proportion of gaze shifts), were measured concurrently with haptic performance during an intermodal word comprehension task. During the task, infants were presented with two images simultaneously on a touchscreen and asked to touch one of the images. Results revealed a somewhat discrepant view on how visual attention relates to haptic performance. Specifically, during trials where an incorrect haptic response was performed, looking-time was significantly greater to the incorrect visual referent, however infants displayed a more sophisticated visual attention style (greater proportion of gaze shifts) known to predict later intellectual functioning. Conversely, during trials where infants failed to perform a haptic response, looking-time to the target was significantly greater than chance however, infants performed significantly fewer gaze shifts. These results will help bridge the gap between literatures that use different response modalities and provide insight on the nature of children's developing knowledge about words.