A gated temporal attention based intra prediction framework for robust deepfake video detection

Area of Science:

• Computer Science
• Artificial Intelligence
• Digital Forensics

Background:

• The proliferation of manipulated videos, particularly deepfakes, poses significant threats to digital security and media authenticity.
• Deepfakes are difficult to detect due to realistic facial movements and smooth frame transitions, necessitating advanced detection techniques.

Purpose of the Study:

• To propose a novel deepfake detection model, IP-GTA Net, that combines intra prediction and temporal attention modeling.
• To enhance the accuracy and robustness of deepfake detection methods.

Main Methods:

• Video frames are divided into blocks and reconstructed using a hybrid convolutional autoencoder to simulate compression and reveal inconsistencies.
• Spatial features are extracted using MobileNetV3, followed by gated convolutional GRU with temporal attention for sequence-wise manipulation detection.

Main Results:

• The IP-GTA Net model achieved 90.52% accuracy and an F1-score of 0.9051 on the Celeb-DF dataset.
• Performance surpassed existing models like XceptionNet, Two-Stream CNN, and EfficientNet-B0.

Conclusions:

• IP-GTA Net offers a promising approach for deepfake detection by integrating intra-frame and temporal analysis.
• The model demonstrates superior performance, contributing to improved media authenticity and digital security.