ON-NSW: Accelerating High-Dimensional Vector Search on Edge Devices with GPU-Optimized NSW

Area of Science:

• Computer Science
• Artificial Intelligence
• Edge Computing

Background:

• Industrial Internet of Things (IIoT) applications increasingly use vector embeddings for AI-driven tasks.
• Efficient approximate nearest neighbor search (ANNS) is crucial for these IIoT workloads.
• Existing graph-based ANNS methods like HNSW are primarily designed for high-end servers, not edge devices.

Purpose of the Study:

• To develop a GPU-optimized HNSW design tailored for edge devices.
• To enhance the performance of ANNS on resource-constrained embedded systems.
• To address the limitations of traditional ANNS methods in edge computing environments.

Main Methods:

• Introduced ON-NSW, a GPU-optimized variant of HNSW for edge devices.
• Utilized a flat graph structure to maximize GPU parallelism.
• Optimized component placement within the NVIDIA Jetson Orin Nano's unified memory.
• Implemented warp-level parallel neighbor exploration and lightweight synchronization.

Main Results:

• ON-NSW achieved up to 1.44x higher throughput compared to original HNSW on NVIDIA Jetson.
• Maintained comparable recall rates to the original HNSW.
• Demonstrated efficient and high-throughput vector search capabilities on embedded edge platforms.

Conclusions:

• ON-NSW offers an effective solution for high-throughput vector search on edge devices.
• The GPU optimization strategy is suitable for real-world high-dimensional datasets in IIoT.
• Enables advanced AI applications on embedded systems through efficient ANNS.