DREAM: A Benchmark Study for Deepfake PhotoRealism AssessMent

Area of Science:

• Computer Vision
• Artificial Intelligence
• Digital Forensics

Background:

• Deepfakes pose a threat to information credibility.
• Existing research primarily focuses on objective deepfake detection, neglecting subjective perception.
• Computational modeling of deepfake photorealism is understudied.

Purpose of the Study:

• To address the lack of research in subjective deepfake perception.
• To introduce a benchmark for automatic photorealism assessment of deepfakes.
• To evaluate deepfake quality and deceptiveness for predicting online influence and improving generation.

Main Methods:

• Development of the DREAM (Deepfake photoREalism AssessMent) benchmark.
• Creation of a diverse deepfake video dataset.
• Large-scale human annotation (140,000 scores, 3,500 annotators) with textual descriptions.
• Evaluation of 18 photorealism assessment methods, including vision-language models and a novel CLIP-based approach.

Main Results:

• The DREAM benchmark provides a comprehensive dataset and evaluation framework.
• Analysis of various photorealism assessment methods, highlighting strengths and weaknesses.
• Insights into deepfake photorealism perception and its computational modeling.

Conclusions:

• The DREAM benchmark and findings lay the groundwork for future research in deepfake photorealism assessment.
• This work facilitates better evaluation of deepfake quality and potential impact.
• The dataset is publicly available to foster further research.