Mixture of Global and Local Experts with Diffusion Transformer for Controllable Face Generation

Area of Science:

• Computer Vision
• Artificial Intelligence
• Generative Models

Background:

• Controllable face generation faces challenges in balancing semantic control and photorealism.
• Existing methods struggle with disentangling attributes and capturing both global structure and local semantics.

Purpose of the Study:

• Introduce Face-MoGLE, a novel framework for advanced controllable face generation.
• Improve fine-grained attribute control and photorealism in generated faces.

Main Methods:

• Semantic-decoupled latent modeling via mask-conditioned factorization.
• Mixture of global and local experts for structural and semantic refinement.
• Diffusion-aware dynamic gating network for evolving denoising processes.
• Creation of multimodal datasets (MM-FFHQ-Female, MM-FairFace-HQ) for evaluation.

Main Results:

• Face-MoGLE outperforms state-of-the-art methods in unimodal and multimodal settings.
• Achieved superior image quality, semantic alignment, and aesthetic preference.
• Generated images are realistic enough to challenge deepfake detection models.
• Demonstrated strong zero-shot generalization capabilities.

Conclusions:

• Face-MoGLE offers a powerful solution for controllable face generation.
• The proposed methods and datasets advance the field of generative AI.
• Highlights the importance of responsible AI development in generative modeling.