CharacterFactory: Sampling Consistent Characters With GANs for Diffusion Models.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Machine Learning

Background:

• Text-to-image models have advanced human-centric generation.
• Existing models struggle with generating consistent, novel identities.
• Need for controllable character creation in digital media.

Purpose of the Study:

• Introduce CharacterFactory, a novel framework for identity-consistent character generation.
• Enable sampling of new characters with stable identities within GAN latent spaces.
• Facilitate seamless integration with diffusion models for diverse applications.

Main Methods:

• Utilize word embeddings of celebrity names as ground truth for identity.
• Train a Generative Adversarial Network (GAN) to map latent space to celebrity embeddings.
• Implement a context-consistent loss for identity preservation across various image contexts.
• Achieve rapid training (10 minutes) and efficient end-to-end inference.

Main Results:

• CharacterFactory demonstrates high performance in identity consistency and editability.
• Generated characters maintain consistent identities across different contexts.
• The framework allows for infinite character sampling during inference.
• Successfully integrates with existing image, video, and 3D diffusion models.

Conclusions:

• CharacterFactory represents a significant advancement in identity-consistent character generation.
• The proposed method offers efficient training and flexible inference for novel character creation.
• Enables new possibilities for digital content creation and character-based applications.