Inv-Adapter: ID Customization Generation via Image Inversion and Lightweight Parameter Adapter.

Area of Science:

• Artificial Intelligence
• Computer Vision

Background:

• Text-to-image generation models have advanced significantly, enabling research in identity (ID) customization.
• Existing personalization methods struggle to balance high-fidelity results with low computational costs.
• A key limitation is the reliance on additional prompt image encoders (e.g., CLIP vision encoder), which can lead to weak alignment and loss of crucial identity information.

Purpose of the Study:

• To develop a novel method, Inv-Adapter, for efficient and high-fidelity ID customization in text-to-image generation.
• To address the limitations of existing methods by eliminating the need for separate image encoders and improving feature alignment.

Main Methods:

• Inv-Adapter utilizes a lightweight parameter adapter to inject ID features into text-to-image models.
• It extracts diffusion-domain representations of ID images using a pre-trained text-to-image model via DDIM image inversion, bypassing the need for an additional image encoder.
• A lightweight attention adapter efficiently embeds these aligned ID prompt features into the base text-to-image model.

Main Results:

• Extensive experiments demonstrate Inv-Adapter's competitiveness across various text-to-image models.
• The method shows high performance in ID fidelity, generation loyalty, speed, and training costs.
• Inv-Adapter also proves effective in terms of model scale and generalization ability in general object scenarios.

Conclusions:

• Inv-Adapter presents a highly competitive and efficient solution for ID customization generation.
• The proposed approach effectively integrates identity features, achieving high fidelity and low costs.
• This method offers significant improvements for personalized image generation tasks.