SGMLN: Sentiment-Guided Mutual Learning Network for Multimodal Sarcasm Detection

Area of Science:

• Computer Science
• Artificial Intelligence
• Natural Language Processing

Background:

• Social media platforms like Twitter exhibit a surge in multimodal sarcastic content.
• Existing sarcasm detection methods often neglect crucial sentiment information and semantic alignment between text and images.

Purpose of the Study:

• To develop an effective model for multimodal sarcasm detection by incorporating sentiment analysis and semantic alignment.
• To enhance the understanding and detection of sarcasm in text and image data.

Main Methods:

• Proposed a sentiment-guided mutual learning network (SGMLN) incorporating a sentiment-guided attention layer.
• Utilized Sentic-BERT for extracting sentiment-aware text vectors and a logistic distribution function for inter-classifier knowledge transfer.
• Focused on semantic alignment between text and image features.

Main Results:

• The SGMLN model demonstrated superior performance in multimodal sarcasm detection compared to existing methods.
• Sentiment-aware representations and semantic alignment significantly improved model effectiveness.
• Mutual learning facilitated knowledge sharing, boosting overall performance.

Conclusions:

• The proposed SGMLN effectively leverages sentiment information and semantic alignment for accurate multimodal sarcasm detection.
• The approach enhances the consistency and understanding between text and image modalities.
• This work offers a promising direction for research in multimodal sentiment analysis and sarcasm detection.