Unifying Multi-Modal Hair Editing via Proxy Feature Blending

Area of Science:

• Computer Vision
• Computer Graphics
• Artificial Intelligence

Background:

• Hair editing is a challenging computer vision task requiring fine-grained control and user-friendly interaction.
• Existing methods using Generative Adversarial Networks (GANs) and diffusion models lack a unified framework for diverse interaction modes and attribute preservation.

Purpose of the Study:

• To develop a novel, unified framework for hair editing that supports multimodal interactions and ensures precise editing with faithful attribute preservation.
• To reformulate hair editing as a proxy-based hair transfer problem.

Main Methods:

• Leveraging StyleGAN's disentangled latent space for precise manipulation and its feature space for attribute preservation.
• Unifying diverse modalities by converting editing conditions into distinct transfer proxies for seamless feature blending.
• Extending the framework to 3D-aware settings using EG3D and PanoHead with multi-view boosted hair feature localization and 3D-tailored proxy generation.

Main Results:

• The proposed method consistently outperforms prior approaches in editing effectiveness, attribute preservation, and visual naturalness.
• Demonstrated superior multi-view consistency in 3D-aware hair editing.
• Achieved unprecedented support for multimodal and mixed-modal hair editing interactions.

Conclusions:

• The novel proxy-based hair transfer paradigm offers a unified and effective solution for complex hair editing tasks.
• The framework successfully decouples editing and preservation objectives, enhancing both precision and fidelity.
• The extension to 3D-aware settings showcases the method's versatility and robustness across different dimensions.