Related Experiment Video
Updated: Mar 15, 2026
07:36
Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects
Published on: November 30, 2018
16.5K
Modulation of scene consistency and task demand on language-driven eye movements for audio-visual integration
1Department of Psychology, National Chengchi University, Taiwan.
Acta Psychologica
|September 19, 2016
Summary
Scene consistency influences word-object mapping during language comprehension. Inconsistent backgrounds draw more attention to target objects, with effects modulated by task demands and timing.
Area of Science:
- Psycholinguistics
- Cognitive Science
- Visual Cognition
Background:
- Spoken language comprehension involves mapping words to visual objects.
- The influence of semantic scene consistency on this mapping process remains unclear.
Purpose of the Study:
- To investigate how scene consistency dynamically affects language-driven eye movements.
- To examine the interplay between spoken input, visual scene semantics, and task demands in audio-visual integration.
Main Methods:
- Utilized the visual world paradigm with two experiments.
- Participants listened to spoken sentences while viewing pictures with consistent, inconsistent, or blank backgrounds for target objects.
- Eye movements were tracked during speech and scene comprehension tasks.
Main Results:
- Fixation proportion was higher for inconsistent target objects compared to consistent ones.
- Task demands modulated the strength and direction of scene inconsistency effects.
- Inconsistency effects varied depending on whether the target word had been mentioned.
Conclusions:
- Spoken language, scene-based knowledge, and task demands interact to guide eye movements during comprehension.
- Semantic consistency of the visual scene plays a dynamic role in word-object mapping.
- Eye-tracking in the visual world paradigm reveals complex audio-visual integration processes.
