BIGPrior: Toward Decoupling Learned Prior Hallucination and Data Fidelity in Image Restoration.

Area of Science:

• Computer Vision
• Machine Learning
• Image Processing

Background:

• Classic image restoration relies on hand-designed priors, limiting performance.
• Deep learning excels but introduces unpredictable hallucinations, hindering adoption.
• Existing methods struggle to separate original data from network-generated content.

Purpose of the Study:

• To develop a novel image restoration framework that overcomes limitations of deep learning.
• To decouple network-prior based hallucination from data fidelity terms.
• To generalize classic restoration algorithms using a Bayesian approach.

Main Methods:

• Introducing the Bayesian Integration of a Generative Prior (BIGPrior) framework.
• Utilizing network inversion to extract image prior information from generative networks.
• Decoupling hallucination and data fidelity terms within a Bayesian framework.

Main Results:

• BIGPrior consistently improves inversion results in image colorization, inpainting, and denoising.
• The framework achieves competitive performance against state-of-the-art supervised methods.
• BIGPrior provides a novel metric for pixel-wise prior reliance versus data fidelity.

Conclusions:

• BIGPrior offers a robust and generalizable approach to image restoration.
• The method effectively reduces hallucinations while maintaining data fidelity.
• BIGPrior represents a significant advancement in integrating deep learning priors with classic restoration techniques.