Localization and detection of deepfake videos based on self-blending method

Area of Science:

• Computer Science
• Artificial Intelligence
• Digital Forensics

Background:

• Deepfake technology poses significant societal security threats due to increasingly realistic manipulated videos.
• Current deepfake detection methods lack generalization and struggle with precise manipulation localization.
• Limited availability of fine-grained annotated datasets hinders research on manipulation localization.

Purpose of the Study:

• To propose a novel spatial-based training method for deepfake video detection without requiring fake samples.
• To enhance the accuracy and generalization capabilities of deepfake detection.
• To improve the precise localization of manipulated regions within deepfake videos.

Main Methods:

• Developed a spatial-based training method combining multi-part local displacement deformation and fusion.
• Generated diverse deepfake feature data and mixed-region labels for localization guidance.
• Utilized the Swin-Unet model with specialized loss functions for detection and localization.

Main Results:

• The proposed method effectively simulates real dataset features without using fake samples.
• Achieved satisfactory detection accuracy on benchmark datasets (FF++, Celeb-DF, DFDC).
• Demonstrated accurate localization of manipulated regions in deepfake videos.

Conclusions:

• The novel self-blending method and Swin-Unet model are effective for deepfake video detection.
• The approach overcomes limitations of existing methods in generalization and localization.
• This research contributes to more robust deepfake forensics and misinformation prevention.