Index Networks.

Area of Science:

• Computer Vision
• Deep Learning
• Image Processing

Background:

• Upsampling operators in convolutional networks (CNNs) often struggle with recovering fine spatial details.
• Existing methods like bilinear interpolation can lead to information loss during the upsampling process.
• Deep image matting highlights the potential of index-guided unpooling for detail recovery.

Purpose of the Study:

• To unify existing upsampling operators using the concept of an index function.
• To introduce a novel 'learning to index' framework for adaptive spatial information handling.
• To develop a flexible, plug-in module (IndexNet) for enhancing CNNs in dense prediction tasks.

Main Methods:

• Unified upsampling operators via the index function concept.
• Introduced an index-guided encoder-decoder framework with adaptive index learning.
• Developed the Index Network (IndexNet) module to dynamically generate indices from feature maps.
• Investigated five families of IndexNet and their application to CNNs with coupled downsampling/upsampling stages.

Main Results:

• Demonstrated the superiority of index-guided upsampling in preserving spatial information over traditional methods.
• IndexNet effectively guides downsampling and upsampling stages without requiring extra supervision.
• Achieved state-of-the-art or competitive performance on image matting, denoising, semantic segmentation, and monocular depth estimation.

Conclusions:

• IndexNet offers a powerful and adaptable solution for enhancing spatial information recovery in CNNs.
• The 'learning to index' paradigm enables dynamic adaptation to local pattern variations.
• IndexNet is broadly applicable to various dense prediction tasks, improving overall network performance.