Temporal Feature Matters: A Framework for Diffusion Model Quantization.

Area of Science:

• Artificial Intelligence
• Computer Vision

Background:

• Diffusion models are powerful for image generation but suffer from slow inference and high memory needs.
• Post-Training Quantization (PTQ) is essential for efficiency, but current methods struggle with diffusion models' timestep-dependent nature.

Purpose of the Study:

• To develop an efficient PTQ framework for diffusion models that addresses challenges with temporal feature preservation.
• To improve the applicability of diffusion models by reducing computational demands without sacrificing generation quality.

Main Methods:

• Introduced a novel quantization framework with three key strategies: Temporal Information Block (TIB)-based Maintenance (TIAR & FSC), Cache-based Maintenance, and Disturbance-aware Selection.
• Developed Temporal Information-aware Reconstruction (TIAR) and Finite Set Calibration (FSC) to maintain temporal features.
• Utilized pre-computation and caching of quantized temporal features to minimize errors and employed a disturbance-aware selection mechanism.

Main Results:

• The proposed framework effectively preserves temporal information crucial for diffusion model denoising.
• Achieved significant reductions in inference time and memory requirements.
• Demonstrated superior performance and acceleration across various datasets, models, and hardware compared to existing methods.

Conclusions:

• The novel quantization framework enables efficient and high-quality image generation using diffusion models.
• The developed strategies successfully mitigate the limitations of traditional PTQ methods for diffusion models.
• This work paves the way for broader adoption of diffusion models in resource-constrained environments.