CoCoCs: co-optimized compressive imaging driven by high-level vision.

Area of Science:

• Optics and Photonics
• Computer Vision
• Machine Learning

Background:

• Compressive imaging (CI) captures high-dimensional data with fewer measurements, requiring reconstruction algorithms.
• Existing CI methods often optimize sensing and reconstruction separately, limiting performance.
• Bridging CI with computer vision and human perception remains a challenge.

Purpose of the Study:

• Introduce CoCoCs, a novel noniterative, end-to-end deep learning framework for compressive imaging.
• Co-optimize optical coding and reconstruction with cascaded computer vision tasks.
• Enhance image and video reconstruction quality for human viewing and machine analysis.

Main Methods:

• Developed a deep learning framework (CoCoCs) for end-to-end optimization.
• Integrated optical coding, image recovery, and computer vision tasks.
• Applied the framework to single pixel imaging and snapshot video compressive imaging systems.

Main Results:

• CoCoCs achieved superior image reconstruction compared to existing methods.
• Evaluations using image quality metrics and mean opinion scores demonstrated realistic outputs.
• Demonstrated improved accuracy in computer vision tasks like image classification and motion recognition.

Conclusions:

• CoCoCs offers a unified framework for compressive imaging, enhancing both reconstruction and downstream vision tasks.
• The framework yields high-quality, human- and machine-friendly images and videos.
• CoCoCs facilitates the integration of compressive imagers with computer vision and perception systems.