A spatiotemporal defect-integrated deepfake video detection and forgery algorithm attribution model

Area of Science:

• Artificial Intelligence
• Computer Vision
• Digital Forensics

Background:

• Deepfake technology presents significant safety concerns due to its use in sophisticated fraudulent activities.
• Existing deepfake detection methods often overlook subtle manipulation traces unique to specific forgery algorithms.
• Attributing deepfakes to their generation algorithms is crucial for understanding forgery types and mitigating misinformation.

Purpose of the Study:

• To propose a spatiotemporal artifact-aware framework for simultaneous deepfake video detection and forgery algorithm attribution.
• To address the limitations of current research by capturing unique manipulation traces and enabling source attribution.

Main Methods:

• Combines Convolutional Neural Networks (CNNs) for local feature learning and Transformers for long-range dependency modeling.
• Integrates frequency-domain filtering to enhance the capture of subtle synthesis traces within convolutional features.
• Utilizes multi-layer outputs (middle and deep) to capture multi-scale forgery traces and fuses predictions for final results.
• Employs a multi-loss optimization strategy (cross-entropy, triplet, hard sample mining) for discriminative feature learning.

Main Results:

• Achieved 97.86±0.18% detection accuracy and 99.81±0.11% AUC on the FaceForensics++ dataset.
• Attained 98.42±0.15% accuracy for forgery algorithm attribution.
• Outperformed most state-of-the-art approaches in both detection and attribution tasks.

Conclusions:

• The proposed framework effectively detects deepfake videos and attributes their generation algorithms.
• The spatiotemporal artifact-aware approach enhances the model's ability to learn robust and discriminative forgery features.
• This work contributes to improved digital security and the fight against AI-generated misinformation.