Related Experiment Video
Updated: Jul 2, 2025
09:11
Revealing Neural Circuit Topography in Multi-Color
Published on: November 14, 2011
15.0K
Multi-Semantic Decoding of Visual Perception with Graph Neural Networks
Rong Li1,2,3, Jiyi Li2,3, Chong Wang2,3
1The Center of Psychosomatic Medicine, Sichuan Provincial Center for Mental Health, Sichuan Provincial People's Hospital, University of Electronic Science and Technology of China, Chengdu 611731, P. R. China.
International Journal of Neural Systems
|February 19, 2024
Summary
This study introduces a new semantic graph learning model to decode multiple semantic categories from brain activity, outperforming existing models. The findings highlight the importance of semantic relationships for visual perception and brain decoding.
Area of Science:
- Neuroscience
- Computer Science
- Cognitive Science
Background:
- Understanding visual perception relies on modeling cortical representations of semantic information.
- Current semantic decoding models often overlook the interactive relationships between objects and prior information.
- The human visual system integrates semantic information from natural scenes.
Purpose of the Study:
- To propose a novel semantic graph learning model for decoding multiple semantic categories from brain activity.
- To investigate the role of inter-category relationships in multi-semantic decoding.
- To provide a computational framework for understanding semantic processing in visual perception.
Main Methods:
- Developed a Graph Neural Network-based semantic graph learning model.
- Validated the model using functional magnetic resonance imaging (fMRI) data from subjects viewing natural images.
- Analyzed brain activity corresponding to 52 semantic categories within 2750 natural images.
Main Results:
- The proposed Graph Neural Network model achieved higher decoding accuracies compared to other deep neural network models.
- A significant correlation was found between the co-occurrence probability of semantic categories and decoding accuracy.
- Hierarchical organization of semantic content in higher visual areas correlated with internal visual experience.
Conclusions:
- The semantic graph learning model offers a superior computational framework for multi-semantic decoding.
- Incorporating semantic relationships enhances the accuracy of decoding visual perception from brain activity.
- This approach supports the understanding of the visual integration mechanism in semantic processing.
Related Concept Videos
01:26Visual System
582
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
582
01:24Vision
53.3K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
53.3K
01:20Parallel Processing
151
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
151
01:12Visual Agnosia
200
Visual agnosia is a condition characterized by the inability to recognize visually presented objects despite having normal vision. For instance, a person with visual agnosia can describe the shape and color of an object but cannot identify or name it. This impairment does not affect their visual field, acuity, color vision, brightness discrimination, language, or memory. An example of this condition in a social setting is someone at a dinner party asking for "that silver thing with a round...
200
01:24Color Vision
578
Color perception begins in the retina, the light-sensitive layer at the back of the eye. Two main theories explain how colors are seen: the trichromatic theory and the opponent-process theory. The trichromatic theory, proposed by Thomas Young in 1802 and extended by Hermann von Helmholtz in 1852, suggests that color vision is based on three types of cone receptors in the retina. These cones are sensitive to different but overlapping ranges of wavelengths corresponding to red, blue, and green.
578
01:15Depth Perception and Spatial Vision
651
Depth perception is the ability to perceive objects three-dimensionally. It relies on two types of cues: binocular and monocular. Binocular cues depend on the combination of images from both eyes and how the eyes work together. Since the eyes are in slightly different positions, each eye captures a slightly different image. This disparity between images, known as binocular disparity, helps the brain interpret depth. When the brain compares these images, it determines the distance to an object.
651