Toward Generalizable Forgery Detection and Reasoning

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Detecting AI-generated images is crucial for combating misuse.
• Existing methods struggle with generalization due to domain gaps among generative models.
• Traditional saliency-based explanations are inadequate for synthesized images.

Purpose of the Study:

• To develop a generalizable and interpretable AI-generated image detection method.
• To unify AI image detection and explanation into a single task (Forgery Detection and Reasoning Task - FDR-Task).
• To leverage Multi-Modal Large Language Models (MLLMs) for improved detection accuracy and reasoning.

Main Methods:

• Introduction of the Multi-Modal Forgery Reasoning dataset (MMFR-Dataset) with 120K images and 378K annotations.
• Proposal of the FakeReasoning framework integrating CLIP and DINO for visual encoding.
• Development of a Forgery-Aware Feature Fusion Module and a Classification Probability Mapper for enhanced MLLM guidance.

Main Results:

• FakeReasoning framework demonstrates robust generalization across multiple generative models.
• The proposed method achieves state-of-the-art performance in both AI image detection and reasoning tasks.
• The unified FDR-Task approach proves effective for accurate and interpretable AI forgery detection.

Conclusions:

• The FakeReasoning framework and FDR-Task offer a significant advancement in detecting and understanding AI-generated images.
• MLLMs, guided by specialized modules, can effectively address the challenges of AI image forensics.
• The MMFR-Dataset facilitates comprehensive evaluation and future research in this domain.