Optimizing Real-Time Object Detection in a Multi-Neural Processing Unit System.

Area of Science:

• Computer Vision
• Hardware Acceleration
• Deep Learning

Background:

• Real-time object detection requires high throughput and low latency.
• Neural Processing Units (NPUs) offer superior energy efficiency and parallel processing for deep learning tasks.
• NPUs are crucial for reducing latency and enhancing speed in real-time applications.

Purpose of the Study:

• To construct a real-time object detection system using YOLOv3 and Neubla's Antara NPU.
• To propose and evaluate performance optimization strategies for NPU-based object detection.
• To analyze the impact of buffering and multi-NPU configurations on system performance.

Main Methods:

• Implemented a real-time object detection system with YOLOv3 on an Antara NPU.
• Employed double buffering to enable parallel data processing between CPU and NPU.
• Utilized queue-based processing for task distribution across multiple NPUs.
• Applied Amdahl's law to analyze performance limitations in multi-NPU environments.

Main Results:

• NPU integration with double buffering improved throughput by 3.35x compared to CPU-only.
• Multi-NPU environments achieved a 4.81x throughput increase.
• Latency was reduced by 1.6x with single/double buffering and 1.18x in multi-NPU setups.
• Object detection accuracy remained high (31.8 mAP on NPU vs. 31.4 mAP on CPU).

Conclusions:

• NPU acceleration, combined with double buffering and multi-NPU task distribution, significantly enhances real-time object detection performance.
• Optimized NPU utilization provides substantial throughput gains and latency reductions without compromising accuracy.
• The proposed methods demonstrate the effectiveness of specialized hardware for demanding AI applications.