Learning Guided Implicit Depth Function With Scale-Aware Feature Fusion.

Area of Science:

• Computer Vision
• Deep Learning
• Image Processing

Background:

• Implicit function learning is popular for single image super-resolution.
• Color-guided depth map super-resolution using implicit functions is less explored.
• Key challenges include feature fusion, scale information integration, and cross-domain attention modeling.

Purpose of the Study:

• To investigate the necessity and applicability of fusing depth and color features in the encoder for continuous upsampling scales.
• To determine the importance of scale information in both encoder and decoder.
• To develop an efficient method for modeling cross-domain feature affinity in the decoder.

Main Methods:

• A transformer-based network with separate depth super-resolution and guidance extraction branches.
• An implicit cross transformer in the encoder fuses color guidance with continuous coordinate mapping and filters irrelevant guidance.
• Scale information is embedded in the encoder's position encoding and feed-forward network for scale-aware representation.
• The decoder uses implicit self-attention and cross-attention for reconstructing high-resolution depth maps.

Main Results:

• The proposed network effectively fuses color and depth features for super-resolution.
• Integrating scale information into the encoder enhances feature representation.
• Experiments demonstrate improved performance on synthetic and real datasets across various upsampling scales.
• The method shows effectiveness for both in-distribution and out-of-distribution scales.

Conclusions:

• The transformer-based approach successfully addresses challenges in color-guided depth map super-resolution.
• Implicit function learning combined with cross-domain attention offers a powerful framework.
• The method provides a significant advancement in generating high-resolution depth maps.