Multimodal false information detection method based on Text-CNN and SE module.

Area of Science:

• Artificial Intelligence
• Computer Science
• Social Media Analysis

Background:

• False information on social media poses societal risks.
• Increasing multimedia content necessitates multimodal detection approaches.
• Existing methods often overlook noise and information loss during multimodal feature fusion.

Purpose of the Study:

• To propose an advanced false information detection model.
• To address limitations in multimodal feature fusion for enhanced accuracy.
• To leverage text and image data effectively for robust detection.

Main Methods:

• Utilized Text-CNN for processing text and image features extracted by BERT and Swin-transformer.
• Employed a modified SE module for fusing multimodal features, reducing noise.
• Incorporated residual network concepts to minimize information loss during fusion.

Main Results:

• Achieved a 6.5% accuracy improvement on the Weibo dataset.
• Demonstrated a 2.0% accuracy improvement on the Twitter dataset compared to attention-based methods.
• Ablation studies confirmed the effectiveness of individual model components.

Conclusions:

• The proposed Text-CNN and SE module-based model significantly enhances false information detection accuracy.
• The method effectively mitigates noise and information loss in multimodal fusion.
• This approach offers a promising solution for combating sophisticated disinformation campaigns.