Easy Tuning of Pre-trained Diffusion Models for One-Step Image Colorization

Recently image colorization has made significant strides by leveraging the generative priors of pre-trained diffusion models. However, the iterative inference required by these models imposes a prohibitive computational burden. In this paper, we demonstrate that one-step image colorization can be achieved with surprising ease by adapting pre-trained text-to-image (T2I) diffusion models in our tailor-made scheme-diverging from current stage-wise training-intensive step distillation techniques. Our key insight is that target color components primarily reside in the low-frequency range and are typically determined during the high-noise stages of iterative denoising. This inspires us to train the denoising mapping exclusively at the maximum noise level (i.e., predicting directly from pure Gaussian noise) conditioned on the grayscale image. While this produces plausible colors in plain regions, it often lacks color vividness in fine-grained structures. To circumvent this, we introduce adversarial training to provide discriminative supervision for the single-step generator, which facilitates stable and rapid convergence. Experimental results demonstrate our superior performance over state-of-the-art methods and strong baselines. Furthermore, we present a comprehensive ablation study of our key design choices, including the adversarial training component, text prompt curation, training scheme, and timestep sampling strategy, to elucidate their respective contributions toward achieving a training-efficient and flexibly controllable one-step colorization model.