Direct Distillation: A Novel Approach for Efficient Diffusion Model Inference.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Machine Learning

Background:

• Diffusion models are state-of-the-art for image generation but suffer from slow, computationally intensive multi-step inference.
• Existing distillation methods often struggle with generative diversity and may implicitly model incorrect posterior distributions.

Purpose of the Study:

• To develop a novel sampling strategy for diffusion models that accelerates inference and reduces computational resource requirements.
• To improve the efficiency and diversity of generated images compared to existing diffusion model sampling techniques.

Main Methods:

• Proposed an information bottleneck to reschedule inference with a lightweight distilled neural network for mapping intermediate stages.
• Validated the approach using COCO and LAION datasets with two distillation models (13.5M and 57.5M parameters).
• Utilized information theory to analyze bottlenecks in existing distillation algorithms and compare performance metrics (FID, CLIP Score).

Main Results:

• Achieved 40-50% reduction in inference steps compared to a stable U-Net diffusion model (859M parameters).
• Significantly reduced multiply-accumulate operations (MACs) per inference step (3954M/3922M vs. 67,749M).
• Obtained a Fréchet Inception Distance (FID) of 16.75 in eight steps, outperforming progressive distillation, adversarial distillation, and DDIM.

Conclusions:

• The proposed information bottleneck and distilled neural network approach effectively accelerates diffusion model inference without compromising image diversity.
• The method offers a significant reduction in computational cost and outperforms existing distillation techniques in terms of FID scores.
• Information theoretic analysis provides insights into diversity issues in current distillation models, highlighting the advantages of the proposed algorithm.