Lightweight Diffusion Models Based on Multi-Objective Evolutionary Neural Architecture Search | JoVE Visualize

Area of Science:

Artificial Intelligence
Computer Vision
Machine Learning

Background:

Diffusion models excel at image generation but suffer from high computational costs and long inference times.
Existing acceleration methods primarily focus on inference steps, neglecting diffusion model architecture optimization.
Optimizing diffusion model architectures is crucial for developing computationally efficient generative models.

Purpose of the Study:

To propose LDMOES (Lightweight Diffusion Models based on Multi-Objective Evolutionary Search), a novel framework for designing efficient UNet-based diffusion models.
To leverage multi-objective evolutionary neural architecture search and knowledge distillation for optimizing diffusion model architectures.
To reduce the computational complexity of diffusion models without compromising image generation quality.

Main Methods:

Implemented a framework combining multi-objective evolutionary neural architecture search with knowledge distillation.
Utilized a modular search space within LDMOES to decouple architecture components and enhance search efficiency.
Validated the proposed method on diverse datasets including CIFAR-10, Tiny-ImageNet, CelebA-HQ, and LSUN-church.

Main Results:

LDMOES achieved approximately 40% reduction in multiply-accumulate operations (MACs) in pixel space, outperforming the teacher model.
On the Tiny-ImageNet dataset, the model generated high-quality images with a competitive FID score of 4.16, demonstrating strong generalization.
In latent space, MACs were reduced by ~50% with negligible performance loss, and nearly 60% reduction in computational cost on LSUN-church.

Conclusions:

LDMOES effectively designs lightweight and efficient UNet-based diffusion models through multi-objective evolutionary search and knowledge distillation.
The proposed method significantly reduces computational costs (MACs) in both pixel and latent spaces while maintaining or improving generation quality.
LDMOES demonstrates strong effectiveness and transferability across various datasets, offering a promising direction for efficient generative AI.