TransCL: Transformer Makes Strong and Flexible Compressive Learning.

Area of Science:

• Computer Vision
• Machine Learning
• Signal Processing

Background:

• Compressive learning (CL) integrates compressed sensing (CS) with machine learning for efficient inference on limited measurements.
• Existing CL methods lack flexibility with fixed CS ratios and struggle with high-resolution (HR) data and complex vision tasks.

Purpose of the Study:

• Introduce TransCL, a novel transformer-based compressive learning framework for large-scale images.
• Enable CL with arbitrary compression ratios and enhance scalability for complex vision tasks.

Main Methods:

• TransCL employs learnable block-based compressed sensing with flexible linear projections for efficient processing of large images in blocks.
• A pure transformer backbone processes CS measurements as sequences, utilizing task-specific heads for various vision tasks.

Main Results:

• TransCL demonstrates strong resistance to interference and adaptability to arbitrary CS ratios.
• Achieved state-of-the-art performance in image classification and semantic segmentation on complex HR datasets.
• Maintained near-original performance at 10% CS ratio and satisfactory results at 1% CS ratio.

Conclusions:

• TransCL offers a flexible and scalable solution for compressive learning on large-scale image data.
• The transformer-based approach significantly advances CL capabilities for real-world vision applications.