Related Experiment Video
Updated: May 16, 2026
07:36
Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects
Published on: November 30, 2018
Can Speaker Gaze Modulate Syntactic Structuring and Thematic Role Assignment during Spoken Sentence Comprehension?
1Cognitive Interaction Technology Excellence Center, Bielefeld University Bielefeld, Germany.
Frontiers in Psychology
|December 11, 2012
Summary
Listeners rapidly follow speaker gaze to anticipate objects, aiding comprehension. This gaze-following effect is robust, influenced by sentence structure, and impacts understanding even after the sentence ends.
Area of Science:
- Psycholinguistics
- Cognitive Science
- Neuroscience
Background:
- Listeners often anticipate upcoming information during comprehension.
- Speaker gaze is a cue that can facilitate predictive processing.
Purpose of the Study:
- To investigate the robustness and limitations of speaker gaze-following during language comprehension.
- To examine how speaker gaze interacts with syntactic structure and sentence complexity.
- To determine the persistence of gaze-following effects on comprehension.
Main Methods:
- Two visual-world eye-tracking experiments using German sentences with varying syntactic structures (SVO, OVS).
- Participants viewed videos of a speaker describing actions between characters, with the speaker's gaze directed at a character or obscured.
- Post-sentence verification tasks assessed referent identification and role-relation understanding.
Main Results:
- Gaze-following led to earlier anticipation of the object (NP2) regardless of sentence structure.
- The effect of gaze-following was more pronounced for Subject-Verb-Object (SVO) than Object-Verb-Subject (OVS) sentences.
- Gaze-following significantly improved the speed of role-relation verification, especially for SVO sentences, and eliminated processing differences between SVO and OVS structures when role relations matched.
Conclusions:
- Speaker gaze-following is a robust mechanism that aids language comprehension, even when the speaker is not directly facing the listener.
- The benefits of gaze-following are modulated by the syntactic structure and thematic roles within a sentence.
- Speaker gaze significantly influences both predictive processing during sentence unfolding and post-sentence comprehension, highlighting its pervasive role in situated language understanding.
Keywords:
eye trackingsentence structurespeaker gazevisual context effectsvisually situated sentence comprehensionMore Related Videos
Related Concept Videos
01:27Language and Cognition
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
01:10Higher Mental Functions of the Brain: Language
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
01:17Auditory Perception
The auditory system is essential for sound perception, utilizing various critical structures. When sound waves enter the outer ear, they travel through the ear canal and cause the eardrum to vibrate. These vibrations are then transmitted to the middle ear, where three tiny bones – the malleus, incus, and stapes – amplify the sound. This amplification is crucial, as it ensures that the sound vibrations are strong enough to be conveyed to the inner ear. These vibrations then reach the cochlea, a...
01:24Components of Language
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs. “eh”). Phonemes combine to...
03:26Framing Effects
Information is everywhere and its presentation—such as how and when items are presented—can impact our perceptions and decisions surrounding the info. This broad concept umbrellas framing effects—influences that occur due to the way information is framed in its appearance, whether it’s purely the order or the specific wording of a message. Let’s take a look at numerous ways in which two versions of something can objectively say the same thing, yet we respond in different ways based on the...
01:24Facial Feedback Hypothesis
Charles Darwin proposed that facial expressions are an evolutionary adaptation for communication. He argued that these expressions are not influenced by culture but are universal across species. For example, a snarling expression with exposed teeth signals a threat in many animals, including humans. Darwin also suggested that displaying an emotion can intensify the feeling. Smiling, for example, could enhance one's sense of happiness. This idea laid the foundation for understanding the role of...