Dual-branch fusion model for lensless imaging

Area of Science:

• Optics and Photonics
• Computer Vision
• Image Processing

Background:

• Lensless cameras offer reduced size, weight, and cost compared to traditional lensed cameras.
• Image reconstruction is a critical challenge in lensless imaging systems.
• Current mainstream reconstruction methods include model-based approaches and data-driven deep neural networks (DNNs).

Purpose of the Study:

• To investigate the strengths and weaknesses of model-based and DNN-based reconstruction methods.
• To propose a novel parallel dual-branch fusion model for enhanced lensless image reconstruction.
• To validate the effectiveness of the proposed model in both simulation and a real-world lensless camera prototype.

Main Methods:

• A parallel dual-branch fusion model is proposed, integrating model-based and data-driven DNN branches.
• Two fusion architectures, Merger-Fusion-Model and Separate-Fusion-Model, are designed, with the latter using an attention module for adaptive weighting.
• A novel UNet-FC network architecture is introduced into the data-driven branch to leverage lensless optics properties.

Main Results:

• The dual-branch fusion model demonstrates superior performance over state-of-the-art methods.
• Quantitative improvements include +2.95dB PSNR, +0.036 SSIM, and -0.0172 LPIPS on a public dataset.
• Experimental validation using a lensless camera prototype confirms the method's effectiveness.

Conclusions:

• The proposed parallel dual-branch fusion model effectively enhances lensless image reconstruction.
• Integrating model-based and data-driven techniques with novel network architectures yields significant improvements.
• The method shows practical applicability in real-world lensless imaging systems.