Image reconstruction with transformer for mask-based lensless imaging.

Area of Science:

• Optics
• Computer Vision
• Machine Learning

Background:

• Lensless imaging offers a compact alternative to traditional cameras but faces challenges in image reconstruction quality.
• Conventional model-based reconstruction methods are limited by system modeling inaccuracies.
• Pure deep neural network (DNN) approaches have not yet surpassed model-based methods for lensless imaging.

Purpose of the Study:

• To improve image reconstruction quality in mask-based lensless imaging.
• To address the limitations of existing DNN reconstruction approaches by incorporating global feature reasoning.

Main Methods:

• Investigated the importance of global features in understanding optically encoded patterns due to the multiplexing property of lensless optics.
• Developed a novel fully connected neural network architecture incorporating a transformer for enhanced global feature reasoning.
• Compared the proposed transformer-based DNN with conventional model-based and fully convolutional network (FCN)-based DNN approaches.

Main Results:

• The proposed transformer-based DNN demonstrated superior global feature reasoning capabilities compared to FCNs.
• The new architecture achieved enhanced image reconstruction quality in lensless imaging.
• Experimental results validated the superiority of the proposed method over existing approaches.

Conclusions:

• Global feature reasoning is crucial for effective image reconstruction in lensless imaging.
• The novel transformer-based DNN architecture represents a significant advancement in lensless imaging reconstruction.
• This approach offers a promising direction for improving the performance of lensless camera systems.