DSA-Diff: Dynamic schedule alignment for training-Inference consistent modality translation in x-prediction diffusion...

Area of Science:

• Artificial Intelligence
• Computer Vision
• Machine Learning

Background:

• Diffusion models excel at image generation, with x-prediction offering advantages over traditional ϵ-prediction.
• Training-inference inconsistency (TII) in x-prediction models stems from mismatches between assumed and true data distributions.
• Existing methods struggle to fully resolve TII, impacting image synthesis fidelity.

Purpose of the Study:

• To propose DSA-Diff, a novel framework addressing training-inference inconsistency (TII) in x-prediction diffusion models.
• To enhance the speed, accuracy, and robustness of image generation for modality translation tasks.
• To improve the fidelity and detail of synthesized images while minimizing computational cost.

Main Methods:

• Developed DSA-Diff framework utilizing dual noise schedules to decouple training and inference.
• Introduced a Bayesian-Greedy Alignment Scheduler (BGAS) for dynamic inference schedule reconstruction.
• Incorporated progressive target prediction and multi-scale perceptual alignment for enhanced model performance.

Main Results:

• DSA-Diff achieves high-fidelity image synthesis in 4-10 adaptive inference steps with low computational cost (68 GFLOPS).
• Demonstrated significant mitigation of TII, improving the SSIM metric by up to 2.56% on the TFW dataset.
• The framework integrates seamlessly with existing x-prediction models via a single algorithmic module.

Conclusions:

• DSA-Diff effectively resolves TII in x-prediction diffusion models, leading to superior image synthesis.
• The proposed method offers a computationally efficient and robust solution for modality translation.
• The framework shows promise for advancing generative AI in computer vision applications.