Hybrid CNN-Mamba network for single-pixel imaging.

Area of Science:

• Computational imaging
• Deep learning for signal processing
• Optics and photonics

Background:

• Deep neural networks like CNNs and ViTs have advanced single-pixel imaging (SPI).
• CNNs struggle with long-range dependencies due to local receptive fields.
• ViTs face computational challenges with high quadratic complexity in attention mechanisms.

Purpose of the Study:

• To develop a novel hybrid network for single-pixel imaging (SPI).
• To overcome limitations of existing deep learning models in capturing long-range dependencies in SPI.
• To improve imaging quality and computational efficiency in SPI reconstruction.

Main Methods:

• Proposed a hybrid network, CMSPI, integrating CNNs and Mamba-based state space models (SSMs).
• Utilized complementary split-concat structure, depthwise separable convolution, and residual connections.
• Implemented a two-step training strategy for enhanced performance and hardware friendliness.

Main Results:

• CMSPI demonstrates higher imaging quality compared to state-of-the-art SPI methods.
• The proposed network exhibits lower memory consumption and reduced computational burden.
• Simulations and real experiments validate the effectiveness of CMSPI.

Conclusions:

• CMSPI effectively models long-range dependencies in single-pixel imaging using a hybrid CNN-SSM approach.
• The network offers a computationally efficient and high-performance solution for SPI.
• CMSPI presents a promising advancement for practical single-pixel imaging applications.