Correlation Verification for Image Retrieval and Its Memory Footprint Optimization.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Conventional image retrieval often relies on geometric re-ranking.
• Existing methods can be computationally expensive, requiring multi-scale inference.
• High memory usage is a limitation for dense feature storage in large-scale retrieval.

Purpose of the Study:

• To introduce a novel image retrieval network, CVNet, that replaces traditional geometric re-ranking.
• To develop an efficient cross-scale matching mechanism.
• To address the memory limitations of CVNet through a sparsification approach.

Main Methods:

• Proposed Correlation Verification Network (CVNet) utilizing a 4D convolutional neural network.
• Implemented feature pyramids for efficient cross-scale feature correlation in a single inference.
• Employed curriculum learning with the Hide-and-Seek strategy for challenging samples.
• Introduced Dense-to-Sparse CVNet with a sparsification module using a Gumbel estimator to reduce memory footprint.

Main Results:

• CVNet achieved state-of-the-art performance on multiple image retrieval benchmarks.
• Dense-to-Sparse CVNet significantly reduced memory usage.
• The sparsification process in Dense-to-Sparse CVNet preserved performance levels comparable to the original CVNet.
• Offline sparsification ensured no increase in online extraction and matching times.

Conclusions:

• CVNet offers a powerful alternative to conventional geometric re-ranking for image retrieval.
• Dense-to-Sparse CVNet effectively mitigates the memory constraints of CVNet, making it practical for real-world applications.
• The proposed sparsification method provides a scalable solution for large-scale image retrieval systems.