Using cascade CNN-LSTM-FCNs to identify AI-altered video based on eye state sequence.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep learning enables advanced capabilities but also facilitates the creation of undetectable DeepFake media.
• DeepFakes are increasingly used for spreading misinformation and false news.

Purpose of the Study:

• To develop a novel method for identifying DeepFake videos and images.
• To alert viewers about potentially false information presented in manipulated media.

Main Methods:

• A hybrid deep learning model combining Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Fully Connected Networks (FCNs).
• Utilized eye-blinking states in temporal video frames as a key indicator for detection.
• Extracted spatial features using VGG16 and temporal features using Long Short-Term Memory (LSTM) networks.
• Developed a new BPD dataset based on pre-processed eye-blinking data for training.

Main Results:

• The proposed model accurately identified tampered videos within the BPD dataset.
• Achieved 90.8% accuracy on the complex FaceForensic++ dataset, demonstrating robustness on unseen data.
• The training process was optimized, reducing computational requirements.

Conclusions:

• The developed deep learning approach effectively detects DeepFake videos by analyzing physiological signals like eye blinking.
• The novel BPD dataset and hybrid model architecture show significant promise for combating disinformation.
• The method offers a reliable tool for verifying media authenticity and mitigating the impact of fake news.