Multiobjective Evolutionary Diffusion Transformer Acceleration

Diffusion Transformers (DiTs) have demonstrated outstanding performance in the field of image synthesis. However, the prohibitive computational overhead of model inference has been a major hurdle for deployment in low-latency scenarios. Based on the repetitive nature of the denoising process, we propose a multiobjective evolutionary algorithm for DiT acceleration, dubbed MEDA. Particularly, the conflict between the computational cost of the sampling process and the degradation in generation quality is explicitly modeled as a two-objective optimization problem, and similar intermediate features in adjacent sampling steps are stored and reused according to evolutionary-optimized schedules. Besides, a dynamic condition evaluation strategy is designed and incorporated into the evolutionary algorithm to maintain the generation performance of accelerated DiT models across different condition inputs. In addition, a latent objective normalization strategy is introduced to provide a bounded objective space for the evolution process. Experiments across various DiT architectures demonstrate that the proposed MEDA can significantly accelerate the sampling process while maintaining or even further enhancing generation quality relative to the original baseline models.