Test-Time Adaptation for Detecting Image Inpainting Forgeries.

Area of Science:

• Computer Science
• Artificial Intelligence
• Image Processing

Background:

• Deep learning-based image inpainting creates highly realistic forgeries, challenging current detection models.
• Detection performance degrades significantly when test data distribution differs from training data (distributional shift).
• Existing methods struggle with the evolving nature and diverse traces of various inpainting techniques.

Purpose of the Study:

• To develop a robust test-time adaptive detection framework for image inpainting forgeries.
• To enhance the adaptability and detection accuracy of forgery detection models in dynamic, out-of-distribution environments.
• To address the limitations of static detection models facing diverse and evolving inpainting methods.

Main Methods:

• Proposing an image gradient-based metric to quantify model uncertainty and guide adaptation.
• Integrating uncertainty metric with sample-specific batch normalization (BN) statistics for enhanced inference.
• Introducing a cross-attention module for dynamic, side-tuning adaptation without altering the backbone network.
• Constructing a diverse dataset of synthetic images from multiple inpainting methods.

Main Results:

• The proposed test-time adaptive framework significantly outperforms existing baseline methods.
• Demonstrated enhanced adaptability of forgery detection models to distributional shifts.
• Achieved improved detection performance in dynamic environments with unseen inpainting variations.
• Validated effectiveness across two scenarios of distributional bias.

Conclusions:

• The developed framework effectively addresses the challenge of detecting sophisticated image inpainting forgeries.
• Test-time adaptation using gradient metrics and cross-attention modules improves model robustness.
• The approach offers a promising solution for maintaining high detection accuracy in real-world, evolving scenarios.