Related Experiment Video
Updated: Jul 4, 2026

07:36
Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects
Published on: November 30, 2018
Describing visual scenes: towards a neurolinguistics based on construction grammar
1Computer Science, University of Southern California, Los Angeles, CA 90089-2520, USA. arbib@usc.edu
Brain Research
|June 20, 2008
Summary
This study models language production and perception using schema competition, exploring Template Construction Grammar (TCG) with SemRep for visual scene descriptions and neurolinguistic applications.
Area of Science:
- Cognitive Science
- Computational Linguistics
- Neuroscience
Background:
- Investigates language production and perception within broader brain mechanisms for action and perception.
- Proposes modeling cognitive functions through schema competition and cooperation.
Purpose of the Study:
- To explore Construction Grammar as a framework for schema-based linguistics using visual scene description.
- To introduce SemRep for representing dynamic visual scenes and generating varied episode descriptions.
- To report on implementing sentence production using Template Construction Grammar (TCG) with SemRep.
Main Methods:
- Utilizes a schema-based approach to model language function.
- Employs Construction Grammar, specifically Template Construction Grammar (TCG).
- Introduces SemRep, a graphical representation for dynamic visual scenes, to SemRep to express semantics in TCG.
Main Results:
- Presents preliminary results on sentence production using TCG and SemRep.
- Demonstrates the potential of SemRep for generating varied descriptions of visual scenes.
- Highlights the suitability of TCG for schema-based linguistic modeling.
Conclusions:
- Suggests that schema-based linguistics, particularly TCG with SemRep, is a viable framework for language production.
- Identifies neural correlates relevant for future neurolinguistic research on TCG.
- Proposes the SemRep-TCG relation as a basis for modeling language comprehension.
More Related Videos
Related Concept Videos
Gestalt Principles of Perception
Gestalt principles provide a framework for understanding how humans perceive objects as unified wholes within their context. These principles are essential in explaining the cognitive processes that make sense of complex visual stimuli by organizing them into coherent groups. One fundamental principle is proximity, which posits that objects located close to each other are perceived as a collective group. For instance, when dots are positioned near one another, the visual system interprets them...
Visual System
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
Vision
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
Components of Language
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs. “eh”). Phonemes combine to...
Language Development
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
Visual Agnosia
Visual agnosia is a condition characterized by the inability to recognize visually presented objects despite having normal vision. For instance, a person with visual agnosia can describe the shape and color of an object but cannot identify or name it. This impairment does not affect their visual field, acuity, color vision, brightness discrimination, language, or memory. An example of this condition in a social setting is someone at a dinner party asking for "that silver thing with a round end"...

