LoRA-Composer: Leveraging Low-Rank Adaptation for Multi-Concept Customization in Training-Free Diffusion Models

Area of Science:

• Artificial Intelligence
• Computer Vision
• Machine Learning

Background:

• Multi-concept customization in image generation is challenging.
• Existing methods using Low-Rank Adaptations (LoRAs) suffer from concept confusion and vanishing.

Purpose of the Study:

• Introduce LoRA-Composer, a training-free framework to seamlessly integrate multiple LoRAs.
• Enhance harmony and preserve distinct characteristics among concepts in generated images.

Main Methods:

• LoRA-Composer employs concept injection and isolation constraints.
• Utilizes an expanded cross-attention mechanism to combat concept vanishing.
• Refines self-attention computation to address concept confusion.

Main Results:

• LoRA-Composer significantly outperforms standard baselines in multi-concept synthesis.
• Demonstrates superior performance, especially in scenarios without image-based conditions.
• Proposed inference techniques accelerate speed and enhance accuracy without performance loss.

Conclusions:

• LoRA-Composer offers an effective solution for training-free multi-LoRA integration.
• The framework successfully addresses key challenges in multi-concept image generation.
• LoRA-Composer advances the field of customizable image synthesis.