Simple Guidance Mechanisms for Discrete Diffusion Models.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Biology

Background:

• Diffusion models excel in continuous data generation but struggle with discrete data due to guidance challenges.
• Existing continuous guidance methods are not directly applicable to discrete diffusion processes.

Purpose of the Study:

• To develop effective guidance mechanisms for controllable discrete diffusion models.
• To introduce a new class of diffusion models leveraging uniform noise for improved guidability and continuous editing.
• To enhance model performance using a novel continuous-time variational lower bound.

Main Methods:

• Derived classifier-free and classifier-based guidance for discrete diffusion.
• Introduced diffusion models utilizing uniform noise for enhanced continuous editing.
• Implemented a novel continuous-time variational lower bound to improve model quality.
• Evaluated models on discrete data domains including genomic sequences, small molecule design, and discretized images.

Main Results:

• Demonstrated improved controllable generation on discrete data compared to autoregressive and diffusion baselines.
• Achieved state-of-the-art performance, particularly with guidance and fast generation settings.
• Showcased the effectiveness of uniform noise diffusion and new guidance mechanisms.

Conclusions:

• The proposed guidance mechanisms and uniform noise diffusion significantly advance controllable generation for discrete data.
• The novel continuous-time variational lower bound contributes to state-of-the-art performance in discrete diffusion models.
• This work offers practical improvements for applications in bioinformatics, drug discovery, and image synthesis.