Multi-Modal Fake News Detection via Bridging the Gap between Modals

Area of Science:

• Artificial Intelligence
• Computer Science
• Information Science

Background:

• Multi-modal fake news detection integrates text and image data.
• Current methods struggle with the semantic gap between modalities.
• Vanilla attention modules inadequately capture cross-modal relationships.

Purpose of the Study:

• To enhance multi-modal fake news detection by addressing the semantic gap.
• To improve the model's ability to capture semantic information from images.
• To optimize the fusion of image and text data for more accurate detection.

Main Methods:

• An image caption-based method was developed to bridge the semantic gap.
• Image description information was integrated into the text representation.
• Combined global and object image features for enhanced representation.
• A transformer model was employed for multi-modal fusion.

Main Results:

• The proposed method significantly improves fake news detection performance.
• Experiments on two public datasets validated the effectiveness of the approach.
• The image caption integration successfully bridged the semantic gap.

Conclusions:

• Image captioning is an effective strategy for enhancing multi-modal fake news detection.
• The proposed method offers a significant improvement over existing techniques.
• Future work can explore more sophisticated fusion mechanisms.