MSFA Image Denoising Using Physics-Based Noise Model and Noise-Decoupled Network.

Area of Science:

• Computer Vision
• Image Processing
• Computational Imaging

Background:

• Multispectral filter array (MSFA) cameras offer compact size and speed but suffer from noise due to narrow-band imaging.
• Existing neural network denoising methods require high-quality paired noisy-clean datasets, which are unavailable for MSFA imaging.
• Accurate noise modeling is crucial for training effective MSFA denoising algorithms.

Purpose of the Study:

• To develop a physics-based noise model for generating realistic noisy images for MSFA denoising.
• To design a novel noise-decoupled neural network architecture for efficient MSFA image denoising.
• To validate the proposed model and network using a real-world MSFA dataset.

Main Methods:

• A physics-based noise model was created, distinguishing between simple (Gaussian, Poisson) and complex (color bias) noise components.
• A noise-decoupled network, comprising SimpleDist noise removal network (SNRNet) and ComplexDist noise removal network (CNRNet), was designed.
• Learnable position embedding was introduced in CNRNet to address non-uniform color bias noise.

Main Results:

• The proposed noise model accurately synthesizes realistic noisy MSFA images.
• The noise-decoupled network trained on synthetic data achieved performance comparable to training on real paired data.
• The developed method outperformed state-of-the-art denoising techniques in experiments.

Conclusions:

• The physics-based noise model effectively simulates MSFA noise characteristics.
• The noise-decoupled network provides a robust solution for MSFA image denoising, even with limited real data.
• This work enables high-quality MSFA image acquisition in light-deficient conditions.