Distributed Adaptive Binary Quantization for Fast Nearest Neighbor Search.

Area of Science:

• Computer Science
• Machine Learning
• Data Mining

Background:

• Hashing is crucial for efficient nearest neighbor search in big data.
• Prototype-based hashing methods offer stronger discriminative power for complex data structures.
• Existing methods like spherical hashing and K-means hashing have limitations in effective binary code utilization.

Purpose of the Study:

• To propose a novel adaptive binary quantization (ABQ) method for enhanced nearest neighbor search.
• To develop a hashing technique that learns discriminative hash functions with unique, small binary codes.
• To address the ineffective coding limitations of current prototype-based hashing methods.

Main Methods:

• Developed an alternating optimization approach to adaptively discover prototype and code sets.
• Proposed an adaptive binary quantization (ABQ) method for learning discriminative hash functions.
• Designed a distributed framework to accelerate large-scale ABQ training.

Main Results:

• The proposed ABQ method significantly outperforms state-of-the-art hashing techniques.
• Experiments on large-scale datasets (up to 80 million) show performance gains up to 58.84%.
• The method demonstrates efficient training, linear to the number of data points, and generalizes to product spaces.

Conclusions:

• Adaptive Binary Quantization (ABQ) provides a robust and efficient solution for nearest neighbor search in big data.
• The method's ability to learn adaptive prototypes and codes leads to superior performance.
• The distributed framework enables scalable application of ABQ to massive datasets.