Related Experiment Video
Updated: Apr 23, 2026

Cross-Modal Multivariate Pattern Analysis
Published on: November 9, 2011
The grammar of visual narrative: Neural evidence for constituent structure in sequential image comprehension
Neil Cohn1, Ray Jackendoff2, Phillip J Holcomb3
1Department of Psychology, Tufts University, 490 Boston Ave, Medford, MA 02155, USA; Department of Cognitive Science, University of California, San Diego, 9500 Gilman Dr. Dept 0526, La Jolla, CA 92093-0526, USA.
Abstract:
Constituent structure has long been established as a central feature of human language. Analogous to how syntax organizes words in sentences, a narrative grammar organizes sequential images into hierarchic constituents. Here we show that the brain draws upon this constituent structure to comprehend wordless visual narratives. We recorded neural responses as participants viewed sequences of visual images (comics strips) in which blank images either disrupted individual narrative constituents or fell at natural constituent boundaries. A disruption of either the first or the second narrative constituent produced a left-lateralized anterior negativity effect between 500 and 700ms. Disruption of the second constituent also elicited a posteriorly-distributed positivity (P600) effect. These neural responses are similar to those associated with structural violations in language and music. These findings provide evidence that comprehenders use a narrative structure to comprehend visual sequences and that the brain engages similar neurocognitive mechanisms to build structure across multiple domains.
More Related Videos
07:36Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects
Published on: November 30, 2018
07:08Investigating Object Representations in the Macaque Dorsal Visual Stream Using Single-unit Recordings
Published on: August 1, 2018
Related Concept Videos
Parallel Processing
Visual System
Once through the pupil, the light passes through the lens, a...
Vision
Gestalt Principles of Perception
Chunking and Rehearsal in Sensory Memory
Structuralism
Titchener's approach to structuralism was unique. He...