SSAT++: A Semantic-Aware and Versatile Makeup Transfer Network With Local Color Consistency Constraint

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Processing

Background:

• Makeup transfer (MT) aims to apply makeup from a reference to a target image while preserving target identity.
• Existing MT methods struggle with semantic correspondence, color fidelity, and temporal consistency in videos, limiting real-world applications.

Purpose of the Study:

• To propose a novel Symmetric Semantic-aware Transfer network (SSAT++) for enhanced makeup transfer.
• To improve makeup similarity, color fidelity, and temporal consistency in video makeup transfer.

Main Methods:

• Introduced a Feature Fusion (FF) module for integrating content and semantic features.
• Developed a Symmetric Mask Semantic Transfer (SMST) module for aligning makeup features based on semantic correspondence.
• Proposed a local color loss for improved color fidelity and a morphing simulation for video temporal consistency.

Main Results:

• SSAT++ demonstrates superior performance in makeup similarity and color fidelity compared to existing methods.
• The proposed method achieves significant improvements in temporal consistency for video makeup transfer, reducing flickering.
• Experiments on MT and MT-Wild datasets validate the effectiveness and flexibility of SSAT++.

Conclusions:

• SSAT++ effectively addresses the limitations of previous makeup transfer techniques.
• The network offers improved realism, semantic accuracy, and temporal stability for both image and video makeup transfer.
• The developed approach provides more flexible control over makeup application.