U-Turn Diffusion.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Diffusion Models

Background:

• Diffusion models generate synthetic data by learning probability distributions.
• The score function (SF) encodes ground truth (GT) sample information within the diffusion process.
• Existing models utilize a lengthy forward and reverse process over artificial time.

Purpose of the Study:

• To investigate GT sample information encoding in the score function.
• To propose and evaluate U-Turn diffusion, an augmented diffusion model with shortened processes.
• To identify critical time points influencing sample fidelity and class representation.

Main Methods:

• Developed U-Turn diffusion, shortening forward/reverse processes to t∈[0→Tu] and t∈[Tu→0].
• Initialized the U-Turn reverse process at Tu using a sample from the forward process distribution.
• Conducted experiments on ImageNet and CIFAR-10 datasets using class-conditioned and multi-class score functions.

Main Results:

• Identified Memorization Time (Tm) where generated samples diverge from GT.
• Discovered Speciation Time (Ts) where samples represent different classes for Tu>Ts>Tm.
• Analyzed score function nonlinearity, showing it becomes effectively affine for t>Ts and approximately affine for t∈[Tm,Ts].

Conclusions:

• U-Turn diffusion effectively shortens diffusion model training and inference.
• Memorization Time and Speciation Time are critical parameters for controlling sample quality and diversity.
• Score function nonlinearity plays a key role in sample characteristics at different time scales.