Real-time target recognition with a hybrid multi-attention transformer-CNN (AttnConvNeXt) for computational ghost...

Area of Science:

• Computational imaging
• Deep learning
• Computer vision

Background:

• Computational ghost imaging (CGI) traditionally requires reconstructed images or pre-processed signals for target recognition.
• Existing CGI classifiers face limitations in handling varying resolutions and low sampling ratios.
• Fourier transforms are often necessary for reconstruction-based recognition, limiting real-time applications.

Purpose of the Study:

• To introduce a unified, multi-resolution, dual-mode computational ghost imaging recognition framework.
• To develop a hybrid deep learning model capable of analyzing raw 1D bucket measurements directly.
• To achieve robust target recognition in CGI under low sampling ratios and in image-free scenarios.

Main Methods:

• A hybrid multi-head attention transformer-CNN (AttnConvNeXt) model was developed.
• The model processes raw 1D bucket measurements directly, bypassing the need for signal reconstruction.
• It integrates multi-head attention with convolutional layers to capture both local and global features across multiple resolutions (128x128, 64x64, 32x32).

Main Results:

• AttnConvNeXt achieved 99%-100% recognition accuracy on reconstructed images across resolutions.
• In image-free mode using only bucket signals, the model achieved 84% accuracy at low sampling ratios (SR<1), outperforming a 12-layer CNN by 65%.
• Real-time performance was demonstrated, with recognition time scaling efficiently from 0.017 s/image (128x128) to 0.00056 s/signal (raw measurements).

Conclusions:

• The AttnConvNeXt model presents the first multi-resolution, dual-mode CGI recognition framework.
• This approach eliminates the need for Fourier transforms in reconstruction-based recognition.
• The framework shows potential for applications in medical diagnostics, low-light surveillance, and imaging through scattering media.