Learned representation-guided diffusion models for large-image generation.

Area of Science:

• Computer Vision
• Machine Learning
• Medical Imaging
• Remote Sensing

Background:

• Diffusion models require auxiliary data for high-fidelity sample synthesis, which is often impractical due to extensive annotation needs in specialized domains.
• Self-supervised learning (SSL) representations capture rich semantic and visual information, potentially serving as proxies for fine-grained human labels.

Purpose of the Study:

• To develop a novel approach for training diffusion models conditioned on SSL embeddings.
• To demonstrate the efficacy of SSL embeddings as substitutes for manual annotations in image generation.
• To enable the synthesis of large, spatially consistent images and explore text-to-image generation.

Main Methods:

• Trained diffusion models using embeddings from self-supervised learning (SSL) models.
• Generated high-quality histopathology and remote sensing images from SSL features.
• Assembled spatially consistent patches from SSL embeddings to construct larger images, preserving long-range dependencies.
• Demonstrated text-to-large image synthesis by generating pathology and satellite images from text descriptions.

Main Results:

• Diffusion models conditioned on SSL embeddings successfully generated high-fidelity histopathology and remote sensing images.
• Generated images augmented real data, improving downstream classifier accuracy for both patch-level and image-scale classification.
• The models exhibited robustness and generalizability, performing well on unseen datasets.
• Successfully synthesized large images from text descriptions, showcasing a new text-to-large image paradigm.

Conclusions:

• SSL representations are effective proxies for fine-grained labels, enabling diffusion model training without manual annotations.
• The proposed method enhances image synthesis quality and improves downstream classification performance.
• The approach is generalizable and robust, with potential applications in various domains requiring high-fidelity image generation.