SimSwap++: Towards Faster and High-Quality Identity Swapping

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• High-quality face identity editing (FIE) in artificial intelligence (AI) content creation faces challenges due to limited high-resolution datasets and model complexity.
• Existing low-resolution FIE methods have advanced, but high-resolution FIE performance remains a bottleneck for practical applications, especially on mobile platforms.

Purpose of the Study:

• To develop a novel, efficient, and high-performance face identity editing approach suitable for mobile platforms.
• To establish a large-scale, high-quality dataset specifically designed for face identity editing tasks.
• To significantly improve model efficiency for high-resolution FIE.

Main Methods:

• Proposing SimSwap++, an upgraded version of SimSwap, incorporating innovations for high-performance model compression.
• Introducing Conditional Dynamic Convolution (CD-Conv), a novel computational primitive addressing the inefficiency of conditional schemes in small models.
• Presenting Morphable Knowledge Distillation (MKD), a heterogeneous distillation method for further model compression and knowledge transfer.

Main Results:

• SimSwap++ achieves state-of-the-art performance with 97.55% identity accuracy on the FaceForensics++ benchmark.
• The proposed method demonstrates extremely low computational complexity at 2.5 GFLOPs.
• SimSwap++ effectively addresses the challenges of dataset limitations and model complexity in high-resolution FIE.

Conclusions:

• SimSwap++ offers a significant advancement in efficient and high-quality face identity editing.
• The developed dataset and novel techniques pave the way for more accessible and powerful AI content creation tools.
• The approach demonstrates the feasibility of deploying sophisticated FIE models on resource-constrained mobile devices.