Dual-Level Cross-Modality Neural Architecture Search for Guided Image Super-Resolution

Area of Science:

• Computer Vision
• Machine Learning
• Image Processing

Background:

• Guided Image Super-Resolution (GISR) reconstructs high-resolution (HR) images using a corresponding HR guidance image from a different modality.
• Current learning-based GISR methods often use symmetric networks and manual fusion strategies, which can overlook modality differences and optimal fusion points.
• Existing approaches face challenges in balancing performance gains with computational complexity.

Purpose of the Study:

• To develop an automated framework for designing efficient and effective Guided Image Super-Resolution (GISR) models.
• To address limitations in existing GISR methods regarding modality differences, feature fusion strategies, and computational efficiency.
• To introduce Neural Architecture Search (NAS) to the GISR domain for automatic model design.

Main Methods:

• Propose a Dual-level Cross-modality Neural Architecture Search (DCNAS) framework.
• Introduce a dual-level search space for identifying optimal architectures and fusion strategies.
• Employ a supernet training strategy with a pairwise ranking loss trained performance predictor to guide the search process.

Main Results:

• The DCNAS framework successfully designed efficient GISR models, including DCNAS-Tiny and DCNAS.
• Discovered models achieved significant performance improvements across multiple GISR tasks: guided depth map super-resolution, guided saliency map super-resolution, guided thermal image super-resolution, and pan-sharpening.
• The study provides insights into effective architectures for cross-modality image super-resolution.

Conclusions:

• DCNAS is the first NAS-based approach for GISR, demonstrating its effectiveness in automating model design.
• The proposed framework offers a novel solution for optimizing GISR models, balancing performance and computational complexity.
• The research opens new avenues for exploring efficient and effective cross-modality image super-resolution techniques.