MIECF: Multi-faceted information extraction and cross-mixture fusion for multimodal aspect-based sentiment analysis

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Computer Vision

Background:

• Multimodal aspect-level sentiment analysis is crucial for AI advancement.
• Existing methods often neglect crucial information from multiple data facets, leading to information loss.
• Focusing on single visual features like facial expressions overlooks valuable textual data within images.

Purpose of the Study:

• To propose a novel approach for Multimodal Aspect-based Sentiment Analysis (MABSA) that overcomes information loss.
• To develop a method that extracts comprehensive visual and textual information from multiple facets.
• To enhance sentiment analysis accuracy through integrated multimodal feature fusion.

Main Methods:

• Introduced Multi-faceted Information Extraction and Cross-mixture Fusion (MIECF) for MABSA.
• Captured comprehensive local (facial expressions, text) and global visual features.
• Designed a Cross-mixture Fusion method to integrate local and global multimodal information, establishing semantic relationships.

Main Results:

• Achieved leading performance with 79.65% accuracy on the Twitter-2015 dataset.
• Obtained Macro-F1 scores of 75.90% (Twitter-2015) and 73.11% (Twitter-2017).
• Demonstrated improved contextual understanding and reduced ambiguity in sentiment analysis.

Conclusions:

• The MIECF approach effectively integrates multi-faceted information for superior MABSA.
• The proposed method enhances sentiment analysis accuracy by leveraging both local and global features.
• This work advances the field of AI by providing a more robust approach to understanding sentiment in multimodal contexts.