MuseumMaker: Continual Style Customization Without Catastrophic Forgetting.

Area of Science:

• Artificial Intelligence
• Computer Vision

Background:

• Text-to-image (T2I) models are increasingly used for customized image generation.
• Catastrophic forgetting hinders T2I models from learning new styles while retaining old ones.

Purpose of the Study:

• To propose MuseumMaker, a method for continuous style synthesis in T2I models.
• To address catastrophic forgetting and overfitting in few-shot style learning.

Main Methods:

• Developed a style distillation loss module to extract and learn new styles, minimizing content bias and overfitting.
• Implemented dual regularization for shared-LoRA to prevent catastrophic forgetting by regularizing weight and feature aspects.
• Introduced task-wise token learning with unique token embeddings for each new style to preserve historical knowledge.

Main Results:

• MuseumMaker successfully synthesizes images following new customized styles without forgetting previously learned ones.
• The method demonstrates robustness and versatility across diverse style datasets.
• Experimental results validate the effectiveness of MuseumMaker in capturing style nuances while preserving details.

Conclusions:

• MuseumMaker offers a solution for continuous style learning in T2I models.
• The proposed method effectively mitigates catastrophic forgetting and overfitting.
• MuseumMaker enables the accumulation of creative artistic works in a 'Museum' format.