SODA: Spectral Orthogonal Decomposition Adaptation for Diffusion Models

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Parameter-efficient adaptation of large generative models is crucial for practical applications.
• Existing methods like low-rank adaptation may limit representation capacity.
• There is a need for adaptation techniques that balance efficiency and performance.

Purpose of the Study:

• To propose a novel spectrum-aware adaptation framework for generative models.
• To introduce Spectral Orthogonal Decomposition Adaptation (SODA) for parameter-efficient fine-tuning.
• To enhance representation capacity without compromising computational efficiency.

Main Methods:

• Developed a framework that adjusts singular values and basis vectors of pretrained weights.
• Utilized Kronecker product and Stiefel optimizers for efficient orthogonal matrix adaptation.
• Introduced Spectral Orthogonal Decomposition Adaptation (SODA).

Main Results:

• SODA demonstrated effectiveness in parameter-efficient adaptation.
• The method balances computational efficiency and representation capacity.
• Evaluations on text-to-image diffusion models confirmed SODA's advantages over existing methods.

Conclusions:

• SODA offers a spectrum-aware alternative for fine-tuning large generative models.
• The proposed method achieves efficient adaptation while preserving high representation capacity.
• SODA shows promise for advancing parameter-efficient model adaptation.