通过多模式网络揭示大脑中的视觉语言集成
Vighnesh Subramaniam1,2, Colin Conwell3, Christopher Wang1,2
1MIT CSAIL.
ArXiv
|July 1, 2024
概括
研究人员使用深度神经网络 (DNN) 识别了整合视觉和语言信息的大脑区域. 他们发现多式模式比单式模式更好地预测大脑活动,精确地确定整合地点.
科学领域:
- 神经科学是一个神经科学.
- 人工智能的人工智能
- 计算神经科学是一种神经科学.
背景情况:
- 了解人类大脑中的多模式集成对于认知科学至关重要.
- 深度神经网络 (DNN) 为分析复杂的神经数据提供了强大的工具.
- 立体脑摄影 (SEEG) 为大脑活动研究提供高分辨率的神经记录.
研究的目的:
- 使用DNN识别参与多式联络 (视觉和语言) 整合的大脑区域.
- 为了比较SEEG录音上的多模式DNN与单模式DNN的预测能力.
- 评估不同的DNN架构和训练技术,以预测神经活动.
主要方法:
- 利用多式深度神经网络 (DNN) 来预测观看电影的受试者的SEEG记录.
- 运营的多式联运一体化站点是多式联运模式优于单式联运或线性联运模式的区域.
- 在具有相同架构和训练集的模型之间进行了受控的比较,只有输入方式有所不同.
主要成果:
- 训练有素的视觉和语言DNN在预测SEEG信号方面明显优于随机初始化的模型.
- 确定了大量的神经部位 (平均为12.94%) 和大脑区域,显示出多模式集成的证据.
- CLIP式的训练成为下游预测这些部位神经活动的最有效的技术.
结论:
- 多式联络DNN是有效的工具,用于发现人脑中多式联络整合的地点.
- 特定的大脑区域表现出显著的多式联络整合能力,如改进的DNN预测所证明的那样.
- 与多式处理相关的神经反应的建模中,CLIP风格的训练显示出卓越的性能.
相关概念视频
Higher Mental Functions of the Brain: Language
786
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
786
Visual System
566
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
566
Lateralization
319
Brain lateralization refers to the division of mental processes and functions between the two hemispheres of the brain, a phenomenon that optimizes neural efficiency and underpins complex abilities in humans. This specialization allows each hemisphere to perform tasks where it has a comparative advantage, facilitating more refined cognitive capabilities across different domains.
319
Vision
53.1K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
53.1K
Motor and Sensory Areas of the Cortex
3.7K
The cerebral cortex, the brain's outermost layer, is pivotal in processing complex cognitive tasks, emotions, and various sensory inputs and executing voluntary motor activities. This intricate structure is divided into three primary functional areas: the motor areas, sensory areas, and association areas.
Motor Areas
The motor areas located in the frontal lobe are central to controlling voluntary movements. This region is further subdivided into the primary motor cortex and the premotor cortex....
Motor Areas
The motor areas located in the frontal lobe are central to controlling voluntary movements. This region is further subdivided into the primary motor cortex and the premotor cortex....
3.7K
Language and Cognition
340
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
340


