AdaGen: Learning Adaptive Policy for Image Synthesis

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Generative models like MaskGIT, diffusion, and flow models decompose synthesis into steps.
• Iterative generation requires configuring step-specific parameters (e.g., mask ratio, noise level).
• Current methods use static, manually-designed schedules, lacking adaptability and requiring expert tuning.

Purpose of the Study:

• Introduce AdaGen, a general, learnable, and sample-adaptive framework for scheduling iterative image generation.
• Address the limitations of static schedules in generative models.
• Enhance performance, flexibility, and efficiency in image synthesis.

Main Methods:

• Formulate scheduling as a Markov Decision Process (MDP).
• Employ a lightweight policy network trained via reinforcement learning to adapt parameters.
• Propose an adversarial reward design for effective policy training.
• Incorporate an inference-time refinement strategy and a fidelity-diversity trade-off mechanism.

Main Results:

• AdaGen demonstrates superiority across five benchmark datasets (ImageNet, MS-COCO, CC3M, LAION-5B) and four generative paradigms.
• Achieves improved performance on DiT-XL with approximately 3x lower inference cost.
• Enhances the FID of VAR from 1.92 to 1.59 with minimal computational overhead.

Conclusions:

• AdaGen offers a flexible and effective approach to adaptive scheduling in iterative generative models.
• The adversarial reward design is crucial for reliable quality and diversity.
• AdaGen significantly improves efficiency and performance in complex image synthesis tasks.