EDA-DM: Enhanced Distribution Alignment for Post-Training Quantization of Diffusion Models

Area of Science:

• Artificial Intelligence
• Computer Vision
• Machine Learning

Background:

• Diffusion models excel at image generation but face latency challenges due to complex networks and lengthy denoising processes.
• Post-training quantization (PTQ) offers a promising approach to compress and accelerate these models without fine-tuning.
• Existing PTQ methods struggle with diffusion models due to dynamic activations, causing distribution mismatch issues.

Purpose of the Study:

• To develop a standardized PTQ method, EDA-DM, to address distribution mismatch in diffusion models.
• To improve the efficiency and performance of quantized diffusion models for real-world applications.
• To enhance the speed and reduce the model size of diffusion models while preserving generation quality.

Main Methods:

• EDA-DM utilizes calibration sample selection guided by latent space feature map density and diversity.
• It optimizes block reconstruction using Hessian loss to align quantized and full-precision model outputs.
• Theoretical analysis informs the optimization of reconstruction at the output level.

Main Results:

• EDA-DM significantly outperforms existing PTQ methods on various models and datasets.
• Achieved 1.83× speedup and 4× compression for Stable-Diffusion on MS-COCO.
• Minimal performance degradation with only a 0.05 loss in CLIP score.

Conclusions:

• EDA-DM effectively resolves distribution mismatch issues in PTQ for diffusion models.
• The proposed method offers a practical solution for deploying diffusion models with reduced latency and complexity.
• EDA-DM demonstrates superior performance and efficiency compared to prior PTQ techniques.