SJD++: Improved Speculative Jacobi Decoding for Training-free Acceleration of Discrete Auto-regressive Text-to-Image...

Area of Science:

• Artificial Intelligence
• Computer Vision
• Machine Learning

Background:

• Autoregressive models generate high-quality images but are slow due to sequential token prediction.
• This slow inference speed limits their practical application in text-to-image generation.

Purpose of the Study:

• To introduce Speculative Jacobi Decoding++ (SJD++), a novel algorithm for accelerating autoregressive text-to-image generation.
• To reduce the number of forward passes required during inference.

Main Methods:

• SJD++ is a training-free probabilistic parallel decoding algorithm.
• It combines Jacobi decoding's multi-token prediction with speculative sampling's drafting-and-verification.
• SJD++ reuses high-confidence draft tokens to further enhance speed.

Main Results:

• Achieved 2x to 3x reduction in inference latency.
• Demonstrated 2x to 7x compression in generation steps.
• Preserved visual quality with no observable degradation.

Conclusions:

• SJD++ significantly accelerates autoregressive text-to-image generation.
• The algorithm offers a practical solution for faster, high-quality image synthesis.
• It provides substantial speedups while maintaining the fidelity of generated images.