ON-NSW: Accelerating High-Dimensional Vector Search on Edge Devices with GPU-Optimized NSW

The Industrial Internet of Things (IIoT) increasingly relies on vector embeddings for analytics and AI-driven applications such as anomaly detection, predictive maintenance, and sensor fusion. Efficient approximate nearest neighbor search (ANNS) is essential for these workloads. Graph-based methods are among the most representative methods for ANNS. However, most existing graph-based methods, such as Hierarchical Navigable Small World (HNSW), are designed for CPU execution on high-end servers and give little consideration to the unique characteristics of edge devices. In this work, we present ON-NSW, a GPU-optimized design of HNSW optimized for edge devices. ON-NSW employs a flat graph structure derived from HNSW to fully exploit GPU parallelism. In addition, it carefully places HNSW components in the unified memory architecture of NVIDIA Jetson Orin Nano. Also, ON-NSW introduces warp-level parallel neighbor exploration and lightweight synchronization to reduce search latency. Our experimental results on real-world high-dimensional datasets show that ON-NSW achieves up to 1.44× higher throughput than the original HNSW on the NVIDIA Jetson device while maintaining comparable recall. These results demonstrate that ON-NSW provides an effective design for enabling efficient and high-throughput vector search on embedded edge platforms.