Deepfake defense: Combining spatial and temporal cues with CNN-BiLSTM-transformer architecture

Area of Science:

• Computer Science, Artificial Intelligence, Machine Learning, Computer Vision, Cybersecurity

Background:

• Deepfakes pose a significant threat to media authenticity and public discourse.
• Existing deepfake detection methods analyzing spatial or temporal features in isolation are insufficient for real-world scenarios.
• Convolutional Neural Networks (CNNs) capture spatial details, while Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) units handle temporal dynamics but struggle with long-term dependencies.

Purpose of the Study:

• To develop a hybrid deep learning architecture for comprehensive spatiotemporal deepfake detection.
• To overcome the limitations of existing models in handling complex video degradation and long-term temporal relationships.

Main Methods:

• A hybrid deep learning framework integrating CNN (MobileNetV2 backbone), bidirectional LSTMs (BiLSTMs), and transformer encoders.
• CNN module for spatial feature extraction, BiLSTM for local temporal motion, and transformer encoder for global temporal relationships.
• Dual-path temporal modeling leveraging sequential learning and attention mechanisms for integrated spatiotemporal analysis.

Main Results:

• The proposed architecture achieved superior performance on FaceForensics++ and DeepFake Detection Challenge (DFDC) datasets.
• Achieved an F1-score of 90.6% and an Area Under the Curve (AUC) of 98.5%, outperforming baseline models.
• Demonstrated strong robustness against various video quality degradations.

Conclusions:

• The hybrid deep learning model offers a practical and scalable solution for accurate deepfake detection.
• The integrated spatiotemporal analysis effectively addresses the limitations of single-feature detection methods.
• The model's robustness makes it suitable for critical applications requiring high-fidelity media verification.