Multimodal Aspect-Based Sentiment Analysis With Plugin-Enhanced Large Language Models.

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Computer Vision

Background:

• Multimodal aspect-based sentiment analysis (MABSA) is challenging due to complex inter-modal connections.
• Existing methods struggle with aligning features from different modalities for MABSA.
• Large language models (LLMs) show promise but face semantic mismatch and high fine-tuning costs.

Purpose of the Study:

• To propose a novel plugin-based approach for MABSA.
• To address limitations of current LLM-based MABSA methods, including semantic mismatch and computational expense.
• To improve the understanding of intricate connections between visual and textual data in MABSA.

Main Methods:

• A plugin-based framework utilizing an attentive graph convolutional network (A-GCN) to encode multimodal knowledge instances.
• Integration of encoded knowledge via a memory-based hub to align representations with LLMs.
• Evaluation on two benchmark MABSA datasets.

Main Results:

• The proposed approach achieves state-of-the-art performance, outperforming existing baselines.
• Demonstrated superior handling of complex connections between multiple modalities.
• Showcased efficient and scalable adaptation of multimodal LLMs for specific MABSA tasks.

Conclusions:

• The plugin-based approach offers a promising solution for MABSA by effectively integrating multimodal knowledge.
• The method enables efficient adaptation of multimodal LLMs, reducing computational costs.
• The approach sets a new standard for MABSA performance and adaptability.