You Only Look One Step: Accelerating Backpropagation in Diffusion Sampling With Gradient Shortcuts.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Diffusion models (DMs) excel at modeling complex data distributions.
• Downstream tasks often require guiding DM generation using differentiable metrics.
• Current methods necessitate computationally expensive full backpropagation during generation.

Purpose of the Study:

• To develop a computationally efficient method for guiding diffusion model generation.
• To reduce the high memory usage and time consumption associated with traditional backpropagation in DMs.
• To enable optimization of downstream metrics without compromising performance.

Main Methods:

• Proposed Shortcut Diffusion Optimization (SDO), a parallel denoising approach.
• Retained the computational graph for only one generation step for gradient propagation.
• Demonstrated SDO's ability to optimize latent spaces and fine-tune network parameters.

Main Results:

• SDO reduces computational costs by approximately 90% compared to full backpropagation.
• Achieved superior performance in tasks like latent space optimization and DM alignment.
• SDO is generic, high-performance, and computationally lightweight.

Conclusions:

• Full backpropagation throughout the entire DM generation process is unnecessary.
• SDO offers a computationally efficient and effective alternative for guided diffusion model generation.
• The method significantly lowers barriers for applying DMs to practical, metric-guided tasks.