A text guided multimodal scale path fusion network for multimodal sentiment analysis.

Area of Science:

• Artificial Intelligence
• Computer Science
• Natural Language Processing

Background:

• Existing multimodal sentiment analysis (MSA) methods struggle with fixed scales and fail to suppress redundant non-linguistic information.
• This limits the dynamic modeling of emotional features and hinders performance improvements.

Purpose of the Study:

• To propose a novel Text-guided Multimodal Scale Path Fusion Network (TMSPF-Net) to address limitations in current MSA methods.
• To dynamically model emotional features across different scales and suppress redundant information in non-linguistic modalities.

Main Methods:

• The TMSPF-Net utilizes a Multi-scale Adaptive Transformer (MAT) to capture intra- and inter-modal interactions at various scales.
• A Text-guided Conflict Elimination Module (TGCEM) filters conflicting signals in non-linguistic modalities using text guidance.
• A Channel Fusion Module integrates features for final sentiment analysis using a pre-trained language model.

Main Results:

• TMSPF-Net demonstrated superior performance across most metrics on the MOSI and MOSEI datasets compared to state-of-the-art methods.
• The model effectively guided the learning of non-linguistic modalities and integrated multi-level sentiment features.

Conclusions:

• TMSPF-Net offers a flexible and effective approach to multimodal sentiment analysis by dynamically integrating multi-scale features.
• The proposed method shows significant potential for advancing sentiment analysis research and applications.