多式变压器增强融合用于语音情绪识别和识别
Yuanyuan Wang1, Yu Gu1, Yifei Yin2
1School of Artificial Intelligence, Xidian University, Xi'an, China.
Frontiers in neurorobotics
|June 7, 2023
概括
这项研究介绍了一种新的多式联络变压器增强融合方法,用于语音情感识别. 这种方法通过有效地整合各种数据来增强多式情感表现,优于现有方法.
科学领域:
- 人工智能的人工智能
- 人与计算机的交互
- 信号处理 信号处理
背景情况:
- 语音情感识别 (SER) 是复杂的,因为主观和模两可的情感表达.
- 多式联网SER方法显示出希望,但在整合异质数据方面存在困难.
- 现有的融合技术往往忽视了模式之间的细粒度相互作用.
研究的目的:
- 开发一种先进的融合策略,以改善多式联络语音情感识别.
- 为应对有效整合来自不同模式的信息的挑战.
- 为了捕捉语音和文本模式内部和之间细粒度的交互.
主要方法:
- 提出了一种使用混合融合策略的多式变压器增强融合方法.
- 实现了一个模型融合模块与交叉变压器编码器,用于多模式情感表示.
- 采用特征级和模型级的融合,以使用文本和多式联络特征进行增强的语音特征提取.
主要成果:
- 与最先进的方法相比,拟议的方法显示出更高的性能.
- 在IEMOCAP和MELD数据集上取得了显著的改进.
- 成功捕获了细粒度的模态相互作用,以更准确地识别情绪.
结论:
- 多模式变压器增强融合方法在为SER集成异质多模式数据方面取得了突破.
- 混合融合策略和交叉变压器编码器有效地增强了多式联络式情感表现.
- 这种方法推进了语音情感识别领域,使得人们能够更细致地理解情感线索.
相关概念视频
Types Of Transformers
1.0K
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
1.0K
Labeling Emotion
189
Emotional labeling is a cognitive process that involves identifying and naming one's emotions, such as anger, fear, happiness, or sadness. It allows individuals to recognize and express their internal emotional states, a critical aspect of emotional regulation and communication. Labeling emotions requires more than mere recognition; it also involves drawing upon memory and contextual cues to understand the current situation and apply a corresponding emotional label. For instance, feeling...
189
The Ideal Transformer
438
In single-phase two-winding transformers, two windings are coiled around a magnetic core characterized by cross-sectional area A and magnetic permeability μ. A phasor current i1 enters the left winding while i2 exits the right winding, establishing the fundamental working of the transformer through electromagnetic principles.
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's...
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's...
438
Multi-input and Multi-variable systems
134
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
134
Transformers
1.1K
A device that transforms voltages from one value to another using induction is called a transformer. A transformer consists of two separate coils, or windings, wrapped around the same soft iron core. However, they are electrically insulated from each other.
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
1.1K
Association Areas of the Cortex
5.6K
Association areas are regions of the cerebral cortex that do not have a specific sensory or motor function. Instead, they integrate and interpret information from various sources to enable higher cognitive processes such as memory, learning, and decision-making. Some key association areas include the following:
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
5.6K


