Discovering Latent Graphs with GFlowNets for Diverse Conditional Image Generation

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Conditional image generation aims to produce diverse outputs, especially when input conditions have inherent uncertainty.
• Existing methods like random seed modification or prompt diversification have limitations in generating meaningful and interpretable diversity.

Purpose of the Study:

• To introduce Rainbow, a novel framework for conditional image generation that effectively addresses uncertainty in conditions/prompts.
• To generate diverse and plausible images by decomposing input conditions into distinct latent representations.

Main Methods:

• Rainbow integrates a latent graph, parameterized by Generative Flow Networks (GFlowNets), into prompt representation computation.
• It utilizes GFlowNets' graph sampling to capture uncertainty and generate multiple trajectories, leading to diverse condition representations and corresponding images.

Main Results:

• Evaluations on natural and medical image datasets show Rainbow improves diversity and fidelity in image synthesis, generation, and counterfactual tasks.
• The framework demonstrates enhanced capability in capturing and reflecting condition uncertainty in generated images.

Conclusions:

• Rainbow offers a novel approach to enhance diversity in conditional image generation by leveraging GFlowNets for latent representation decomposition.
• The framework is broadly applicable to pretrained conditional generative models, improving performance across various image generation tasks.