AIA-Net: Adaptive Interactive Attention Network for Text-Audio Emotion Recognition.

Area of Science:

• Artificial Intelligence
• Human-Computer Interaction
• Natural Language Processing

Background:

• Emotion recognition is crucial for natural human-computer interaction.
• Current multimodal fusion strategies overlook varying modality importance.
• Effective fusion algorithms considering auxiliary structures are challenging.

Purpose of the Study:

• To propose an adaptive interactive attention network (AIA-Net) for multimodal emotion recognition.
• To address the challenge of unequal modality roles in emotion recognition.
• To enhance fusion algorithms by treating text as primary and audio as auxiliary modalities.

Main Methods:

• Developed an adaptive interactive attention network (AIA-Net).
• Treated text as the primary modality and audio as the auxiliary modality.
• Utilized interactive attention weights and collaborative learning layers for multimodal interaction.

Main Results:

• AIA-Net adapts to different feature dimensions and learns dynamic interactive relations.
• The network effectively focuses on acoustic features beneficial for textual emotion representation.
• Experimental results show superior performance over state-of-the-art methods on benchmark datasets.

Conclusions:

• AIA-Net demonstrates significant effectiveness in multimodal emotion recognition.
• The proposed method enhances textual emotional representation using acoustic information.
• The network facilitates deep, bottom-up evolution of emotional representations through co-learning.