PDGrad: Guiding Diffusion Model for Reference-Based Blind Face Restoration with Pivot Direction Gradient Guidance

Area of Science:

• Computer Vision
• Artificial Intelligence

Background:

• Reference-based blind face restoration (RefBFR) uses reference images to restore degraded faces from unknown factors.
• Guided diffusion models show promise in RefBFR without training by integrating multiple loss gradients.

Purpose of the Study:

• To introduce Pivot Direction Gradient guidance (PDGrad), a novel gradient adjustment method for RefBFR.
• To address the issue of sub-optimal results caused by conflicting gradients in existing guided diffusion models.

Main Methods:

• Developed a loss function incorporating low-level and high-level features, integrating both restored and reference images.
• Introduced a pivot gradient for each feature level to resolve gradient conflicts.
• Implemented adaptive scaling for gradients exceeding the pivot gradient's magnitude.

Main Results:

• PDGrad effectively manages conflicting gradients from multiple losses.
• The method ensures maximization of feature strengths from both restored and reference images.
• Extensive experiments on the CelebRef-HQ dataset demonstrated significant quantitative and qualitative improvements.

Conclusions:

• PDGrad offers a superior approach to RefBFR within guided diffusion frameworks.
• The method effectively balances information from multiple sources, leading to enhanced face restoration.
• PDGrad outperforms existing methods, showing its potential for real-world applications.