REMAP: Multi-layer entropy-guided pooling of dense CNN features for image retrieval

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Large-scale image retrieval faces challenges with variations in illumination, scale, occlusion, and background clutter.
• Existing methods struggle with robustness across diverse and extensive image datasets.

Purpose of the Study:

• To develop a robust and accurate image retrieval system for very large-scale datasets.
• To introduce a novel Convolutional Neural Network (CNN)-based global descriptor that enhances search robustness.

Main Methods:

• Proposed REMAP, a novel CNN-based global descriptor learning and aggregating a hierarchy of deep features from multiple CNN layers.
• Employed end-to-end training with a triplet loss, incorporating spatial max-pooling within multi-scale regions.
• Utilized KL-divergence to measure information gain for relative entropy-guided aggregation, weighting regions and layers based on semantic usefulness.

Main Results:

• REMAP descriptor achieved state-of-the-art mean Average Precision (mAP) on benchmark datasets: 95.5% (Holidays), 91.5% (Oxford), and 80.1% (MPEG).
• Demonstrated superior performance compared to classical CNN-based aggregation methods controlled by Stochastic Gradient Descent (SGD).
• REMAP was the core component of the winning submission for the Google Landmark Retrieval Challenge on Kaggle.

Conclusions:

• The proposed REMAP descriptor significantly improves accuracy and robustness in large-scale image retrieval.
• Relative entropy-guided aggregation effectively weights features, outperforming standard aggregation techniques.
• REMAP represents a significant advancement in content-based image retrieval, particularly for landmark recognition.