A spatial-frequency hybrid restoration network for JPEG compressed image deblurring

Image deblurring and compression-artifact removal are both ill-posed inverse problems in low-level vision tasks. So far, although numerous image deblurring and compression-artifact removal methods have been proposed respectively, the research for explicit handling blur and compression-artifact coexisting degradation image (BCDI) is rare. In the BCDI, image contents will be damaged more seriously, especially for edges and texture details. Therefore, the restoration of the BCDI is a more severe ill-posed inverse problem, and deep mining of local and global feature information is critical for effective BCDI restoration. To this end, we propose a spatial-frequency hybrid restoration network (SFHRN) for explicit and effective joint-photographic-experts-group (JPEG) compressed BCDI restoration. Specifically, according to the nature of JPEG compression artifacts, we propose a spatial-frequency hybrid block (SFHB), which includes a dual-branch structure and an information screening strategy (ISS). First, for the dual-branch structure, we design a patch-level channel attention branch (PCAB) and a pixel-level global attention branch (PGAB) to fully exploit local context information in the spatial domain and mine the global feature information in the frequency domain respectively. Secondly, we design a simple and effective information screening strategy (ISS) to discriminatively determine which pixels and channels should be retained and enhanced in frequency and spatial domains respectively for latent clear image restoration. Finally, for the first time, we build the blur and compression-artifact coexisting degradation datasets by adding various degrees of JPEG compression-artifact into existing benchmark deblurring datasets, e.g. GoPro and HIDE, named as GoPro-Compressed and HIDE-Compressed respectively. Extensive experiments demonstrate the superiority of our proposed SFHRN in terms of both performance and computational cost.