Dual-Ascent-Inspired Transformer for Compressed Sensing.

Area of Science:

• Computer Vision
• Machine Learning
• Signal Processing

Background:

• Deep learning models for image compressed sensing (CS) typically require pre-training for specific compression ratios, limiting their flexibility.
• Traditional iterative algorithms offer flexibility but often lack the efficiency of deep learning approaches.

Purpose of the Study:

• To develop a novel deep learning architecture for image CS that maintains stable performance across diverse compression ratios with minimal training costs.
• To leverage the mathematical properties of the dual ascent method (DAM) for accelerated training convergence in CS models.

Main Methods:

• Proposed the Dual-Ascent-Inspired Transformer (DAT), a novel architecture incorporating dual ascent method principles.
• Introduced an asymmetric primal-dual space within iteration layers for dimension-specific operations, balancing reconstruction quality and efficiency.
• Optimized the Cross Attention module via parameter sharing to reduce training complexity.

Main Results:

• DAT demonstrated superior performance across multiple compression ratios (10%, 30%, 50%) within early-stage training (≤10 epochs).
• Achieved comparable Peak Signal-to-Noise Ratio (PSNR) to ISTA-Net+ within one epoch, significantly faster than competing methods.
• Showcased enhanced robustness to variations in initial learning rates during training.

Conclusions:

• The DAT architecture offers a flexible and efficient solution for image compressed sensing, overcoming the limitations of fixed-ratio pre-trained models.
• DAT's dual ascent-inspired design accelerates convergence and improves performance stability across different compression scenarios.
• The model provides a promising direction for developing adaptable and computationally efficient deep learning-based CS systems.