Eye fixations of deaf and hearing observers in simultaneous communication perception

The purpose of this study was to examine characteristics of eye gaze behavior, specifically eye fixations, during reception of simultaneous communication (SC). SC was defined as conceptually accurate and semantically based signs and fingerspelling used in conjunction with speech. Specific areas of focus were (1) the pattern of frequency, duration, and location of observers' eye fixations in relation to the critical source of disambiguating information (signs or speech) in SC, and (2) how the pattern of an observer's eye fixations was related to the source of critical information (sign or speech), expectations regarding the location of the critical information after exposure to the stimulus set, observer characteristics, and sender.

The investigation used eye tracking technology to monitor eye fixations of observers who watched silent video clips of sentences rendered in SC by three senders. Each sentence contained one of a pair of sign-critical (e.g., "sleeves"/"leaves") or speech-critical (e.g., "invited"/"hired") contrast items designed to depend on information at the hands or mouth, respectively, to resolve its ambiguity. Observers were 20 adults: five faculty/staff with early onset deafness, five faculty/staff with normal hearing, and ten college students with early onset deafness. Faculty and staff were identified by a sign language assessment specialist to be experienced and skillful users of SC. Students, exposed to SC in classroom instruction, were recruited through paper and electronic ads.

Generally, observers looked toward the face, regardless of whether signs or speech disambiguated the message, suggesting that eye fixations toward the hands of the sender are not necessary to apprehend essential information to accurately identify an ambiguous part of the message during SC. However, other aspects of eye behavior indicated sensitivity to type of critical contrast. In particular, fixations were shorter during sign-critical items compared to speech-critical items, even after adjusting for stimulus length. In addition, experienced, adult deaf users of SC made more, brief eye fixations than observers who had normal hearing. Finally, differences in eye fixation patterns toward different senders indicates that sender characteristics affect visual processes in SC perception.

This study provides supportive evidence of brief, frequent eye movements by deaf perceivers over small areas of a video display during reception of visuospatial linguistic information. These movements could be used to enhance activation of brain centers responsible for processing motion, consistent with neurophysiological evidence of attentional mechanisms or visual processes unique to perception of a visual language.