Identifying multimodal misinformation leveraging novelty detection and emotion recognition.

Area of Science:

• Computer Science
• Artificial Intelligence
• Information Science

Background:

• Multimodal fake news, using real media with manipulated text, is prevalent on social media.
• Detecting this misleading content is challenging due to its reliance on external context and novel, emotion-invoking elements.

Purpose of the Study:

• To address the challenge of detecting multimodal fake news.
• To introduce a novel dataset and a multimodal framework for enhanced fake news detection.

Main Methods:

• Developed a new multimodal fake news dataset incorporating background knowledge from authentic sources.
• Designed a multimodal framework employing Supervised Contrastive Learning (SCL) for novelty detection and emotion prediction tasks.

Main Results:

• The proposed framework demonstrated superior performance compared to existing state-of-the-art (SOTA) models in detecting multimodal fake news.
• Extensive experiments validated the effectiveness of the SCL-based novelty detection and emotion prediction components.

Conclusions:

• The novel dataset and multimodal framework offer a promising solution for identifying sophisticated forms of online misinformation.
• Integrating background knowledge and analyzing content novelty and emotion are crucial for robust fake news detection.