Parallel Diffusion Solver via Residual Dirichlet Policy Optimization

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Diffusion models (DMs) excel at generative tasks but are limited by slow sampling speeds due to sequential denoising.
• Existing acceleration methods often degrade image quality, especially under low-latency constraints, due to accumulated errors in capturing complex data distributions.

Purpose of the Study:

• To develop a novel Ordinary Differential Equation (ODE) solver, the Ensemble Parallel Direction solver (EPD-Solver), to accelerate diffusion model sampling.
• To mitigate image quality degradation in accelerated diffusion models by addressing truncation errors in high-curvature trajectory segments.
• To enhance performance in text-to-image generation tasks through parameter-efficient fine-tuning using Reinforcement Learning (RL).

Main Methods:

• Proposed EPD-Solver, an ODE solver that uses multiple parallel gradient evaluations per step to reduce truncation errors.
• Leveraged the Mean Value Theorem for vector-valued functions for more accurate integral approximation of sampling trajectories.
• Introduced a two-stage optimization: initial distillation-based parameter learning followed by parameter-efficient RL fine-tuning for solver optimization.

Main Results:

• EPD-Solver achieved state-of-the-art FID scores at 5 NFE (Number of Function Evaluations): 4.47 (CIFAR-10), 7.97 (FFHQ), 8.17 (ImageNet), and 8.26 (LSUN Bedroom).
• The RL-tuned EPD-Solver significantly improved human preference scores on Stable Diffusion v1.5 and SD3-Medium text-to-image generation.
• Outperformed the official 28-step SD3-Medium baseline with only 20 steps, demonstrating effective inference efficiency and high-fidelity generation.

Conclusions:

• EPD-Solver effectively accelerates diffusion model sampling while maintaining or improving image quality, addressing limitations of previous methods.
• The parameter-efficient RL fine-tuning scheme enhances performance on complex text-to-image tasks without extensive computational overhead.
• EPD-Solver offers a flexible, plugin-based solution (EPD-Solverplugin) to improve existing ODE samplers for diffusion models.