外交部-CNN:一种情绪识别网络集成1D-2D卷积神经网络和跨模态因果特征
Jing Zhang1, Anhong Wang1, Suyue Li1
1School of Electronic Information Engineering, Taiyuan University of Science and Technology, Taiyuan 030024, China.
Brain sciences
|November 27, 2025
概括
这项研究引入了一种新的情绪识别框架,使用脑电图 (EEG) 和功能近红外光谱 (fNIRS) 信号. 整合格兰杰因果关系通过揭示大脑活动相互作用来提高准确性.
科学领域:
- 情感计算是一种情感计算.
- 神经科学是一个神经科学.
- 生物医学工程 生物医学工程
背景情况:
- 像EEG和fNIRS这样的生理信号是理解情感计算中的大脑机制的关键.
- 当前的研究往往忽视了EEG和fNIRS信号之间的因果关系,而是专注于特征或决策层面的融合.
研究的目的:
- 开发一种新的情绪识别框架,使用同时采集EEG和fNIRS.
- 调查EEG和fNIRS信号之间的因果关系,以提高情绪识别.
主要方法:
- 在卷积神经网络 (CNN) 中,提出了一个整合格兰杰因果关系 (GC) 与模态频率注意力机制 (MFA) 的框架.
- 使用GC来理解神经电和血液动力相互作用的EEG和fNIRS信号之间的量化因果关系.
- 开发了一个1D2D-CNN框架,其中有一个MFA模块,用于融合时间,空间和跨模式信息.
主要成果:
- 拟议的方法在情绪识别任务中显著超过了现有的基线.
- 证明了结合来自EEG-fNIRS相互作用的因果特征的有效性.
- 在单模和多模识别场景中实现了卓越的性能.
结论:
- 将基于GC的交叉模式因果特征与模式频率注意力结合起来,可以改善EEG-fNIRS情绪识别.
- 该框架为理解与情绪相关的大脑活动提供了更具生理解释性的方法.
- 强调探索因果关系对于推进情感计算的重要性.
相关概念视频
01:20Parallel Processing
609
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
609
01:21Association Areas of the Cortex
8.7K
Association areas are regions of the cerebral cortex that do not have a specific sensory or motor function. Instead, they integrate and interpret information from various sources to enable higher cognitive processes such as memory, learning, and decision-making. Some key association areas include the following:
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
8.7K
01:24Prosopagnosia
671
Prosopagnosia, also known as face blindness, is the inability to recognize faces. In severe cases, individuals with prosopagnosia may not recognize close family members, including parents and spouses, by their faces. For instance, someone with prosopagnosia might walk past their child in a crowd, only realizing their mistake upon noticing their child's distinctive backpack or favorite jacket. Prosopagnosia specifically impairs facial recognition, while the recognition of other objects or...
671
01:24Vision
59.2K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
59.2K
01:14Motor and Sensory Areas of the Cortex
6.8K
The cerebral cortex, the brain's outermost layer, is pivotal in processing complex cognitive tasks, emotions, and various sensory inputs and executing voluntary motor activities. This intricate structure is divided into three primary functional areas: the motor areas, sensory areas, and association areas.
Motor Areas
The motor areas located in the frontal lobe are central to controlling voluntary movements. This region is further subdivided into the primary motor cortex and the premotor cortex....
Motor Areas
The motor areas located in the frontal lobe are central to controlling voluntary movements. This region is further subdivided into the primary motor cortex and the premotor cortex....
6.8K
01:26Visual System
1.6K
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
1.6K