CoCoFR: Collaborative codebooks learning with soft matching strategy for blind face restoration.

Area of Science:

• Computer Vision
• Image Processing
• Artificial Intelligence

Background:

• Blind Face Restoration (BFR) aims to recover high-quality (HQ) facial images from degraded versions.
• Existing BFR methods utilize priors like codebooks but face limitations.
• Single spatial-domain codebooks and suboptimal feature matching hinder performance.

Purpose of the Study:

• To address limitations in current BFR methods.
• To propose CoCoFR, a novel approach for superior blind face restoration.
• To improve facial image quality by leveraging multi-domain information and adaptive feature matching.

Main Methods:

• CoCoFR learns collaborative codebooks in both spatial and frequency domains.
• A Dual Codebooks Cross Attention (DCCA) module enables adaptive matching between low-quality (LQ) and HQ features.
• A Mamba-based fusion block (MFB) facilitates coarse-to-fine feature fusion.

Main Results:

• CoCoFR demonstrates superior performance compared to state-of-the-art methods.
• Experiments on synthetic and real-world datasets validate the method's effectiveness.
• The approach achieves higher realness and fidelity in restored facial images.

Conclusions:

• CoCoFR effectively overcomes the limitations of existing BFR techniques.
• The proposed method offers significant improvements in facial image restoration.
• CoCoFR represents a promising advancement in the field of blind face restoration.