Lightweight Diffusion Models Based on Multi-Objective Evolutionary Neural Architecture Search

Area of Science:

• Artificial Intelligence
• Computer Vision
• Machine Learning

Background:

• Diffusion models excel at image generation but suffer from high computational costs and long inference times.
• Existing acceleration methods primarily focus on inference steps, neglecting diffusion model architecture optimization.
• Optimizing diffusion model architectures is crucial for developing computationally efficient generative models.

Purpose of the Study:

• To propose LDMOES (Lightweight Diffusion Models based on Multi-Objective Evolutionary Search), a novel framework for designing efficient UNet-based diffusion models.
• To leverage multi-objective evolutionary neural architecture search and knowledge distillation for optimizing diffusion model architectures.
• To reduce the computational complexity of diffusion models without compromising image generation quality.

Main Methods:

• Implemented a framework combining multi-objective evolutionary neural architecture search with knowledge distillation.
• Utilized a modular search space within LDMOES to decouple architecture components and enhance search efficiency.
• Validated the proposed method on diverse datasets including CIFAR-10, Tiny-ImageNet, CelebA-HQ, and LSUN-church.

Main Results:

• LDMOES achieved approximately 40% reduction in multiply-accumulate operations (MACs) in pixel space, outperforming the teacher model.
• On the Tiny-ImageNet dataset, the model generated high-quality images with a competitive FID score of 4.16, demonstrating strong generalization.
• In latent space, MACs were reduced by ~50% with negligible performance loss, and nearly 60% reduction in computational cost on LSUN-church.

Conclusions:

• LDMOES effectively designs lightweight and efficient UNet-based diffusion models through multi-objective evolutionary search and knowledge distillation.
• The proposed method significantly reduces computational costs (MACs) in both pixel and latent spaces while maintaining or improving generation quality.
• LDMOES demonstrates strong effectiveness and transferability across various datasets, offering a promising direction for efficient generative AI.