STFANet: A spatial and temporal feature aggregation network for fake face detection in videos

Area of Science:

• Computer Vision
• Artificial Intelligence
• Digital Forensics

Background:

• Rapid advancements in video synthesis technologies make video authenticity verification increasingly difficult.
• Current detection methods often overlook comprehensive spatio-temporal characteristics, relying mainly on intra-frame spatial artifacts and temporal inconsistencies.
• This limitation restricts the ability to effectively detect sophisticated video manipulations.

Purpose of the Study:

• To propose a novel network, the Spatial and Temporal Feature Aggregation Network (STFANet), for enhanced video forgery detection.
• To address the limitations of existing methods by effectively exploiting spatio-temporal features.
• To improve the accuracy and robustness of manipulated video detection, particularly for facial forgeries.

Main Methods:

• Developed a two-path network structure within STFANet to independently extract spatial and temporal features.
• Integrated extracted spatial and temporal features to create high-fidelity spatio-temporal representations.
• Incorporated a Vision Transformer module to capture global dependencies within feature maps for enhanced representation.

Main Results:

• STFANet demonstrated high efficacy in detecting facial forgeries in videos.
• Achieved excellent performance on benchmark datasets: an AUC score of 0.9933 on FaceForensics++ and 0.9829 on Celeb-DF.
• Analysis confirmed that feature aggregation significantly improves the quality of spatio-temporal representations.

Conclusions:

• The proposed STFANet effectively addresses the challenges in video authenticity verification.
• The network's ability to aggregate spatial and temporal features, enhanced by a Vision Transformer, leads to superior forgery detection performance.
• STFANet represents a significant advancement in detecting manipulated videos, particularly facial forgeries.