The Devil is in the Upsampling: Architectural Decisions Made Simpler for Denoising with Deep Image Prior

Area of Science:

• Computer Vision
• Machine Learning
• Image Processing

Background:

• Deep Image Prior (DIP) demonstrates inherent image smoothing and noise resistance due to spectral bias in network architectures.
• Image denoising is a primary application, reducing reliance on extensive training datasets.
• Practical challenges include architectural design and noise fitting, often addressed by manual crafting or extensive architecture search.

Purpose of the Study:

• To understand the underlying mechanisms of DIP's denoising capabilities from a frequency perspective.
• To develop straightforward strategies for selecting optimal architectures for image denoising.
• To improve denoising performance and reduce model complexity compared to existing methods.

Main Methods:

• Analysis of DIP's spectral bias through a frequency domain lens.
• Identification of unlearnt upsampling as the critical component driving denoising.
• Development of image-specific architecture selection strategies based on frequency analysis.

Main Results:

• Demonstrated that unlearnt upsampling is the primary driver of DIP's denoising effect.
• Proposed simple strategies for architecture selection, eliminating the need for laborious search.
• Achieved superior denoising results with significantly fewer parameters (up to 95% reduction) compared to existing methods.

Conclusions:

• Unlearnt upsampling is the fundamental mechanism behind DIP's denoising efficacy.
• Image-specific architecture selection based on frequency properties offers an efficient and effective approach.
• The proposed under-parameterized architectures exhibit reduced susceptibility to noise fitting, enhancing robustness.