Semantic representation learning for a mask-modulated lensless camera by contrastive cross-modal transferring

Area of Science:

• Computational imaging
• Artificial intelligence in optics
• Machine learning for scientific imaging

Background:

• Lensless computational imaging combines optical measurements with algorithms.
• Traditional methods require costly supervised learning due to ill-posed measurements.
• Artificial neural networks offer new possibilities for lensless imaging.

Purpose of the Study:

• To develop a self-supervised learning method for lensless imaging.
• To learn semantic representations from implicitly provided priors.
• To overcome the limitations of costly supervised approaches in lensless imaging.

Main Methods:

• A self-supervised learning approach using a contrastive loss function.
• Training a target extractor (measurements) from a source extractor (natural scenes).
• Transferring cross-modal priors into a shared latent space.

Main Results:

• The method effectively classifies mask-modulated scenes on unseen datasets.
• Achieved comparable accuracy to the contrastive language-image pre-trained (CLIP) network.
• Demonstrated the transferability of priors from structured natural scenes to modulated scenes.

Conclusions:

• The proposed method avoids costly data annotation for lensless imaging.
• It offers greater adaptability to unseen data compared to conventional techniques.
• The multimodal representation learning is applicable to various downstream vision tasks in unconventional imaging settings.