Object detection on low-compute edge SoCs: a reproducible benchmark and deployment guidelines.

Area of Science:

• Computer Science
• Artificial Intelligence
• Embedded Systems

Background:

• Deploying deep learning object detectors on edge AI Systems-on-Chips (SoCs) faces challenges due to factors beyond theoretical compute ratings.
• Real-world performance is influenced by architectural design, memory bandwidth, and system-level contention.

Purpose of the Study:

• To provide a comprehensive and reproducible benchmark of nine YOLO object detection variants on three Rockchip SoCs.
• To analyze performance across various input resolutions, compute configurations, and operating conditions.

Main Methods:

• Benchmarking nine YOLO variants on three Rockchip SoCs.
• Evaluating performance metrics including inference latency, detection accuracy (mAP), FLOPs, parameter count, and energy-per-inference.
• Testing under different input resolutions, compute settings, and multitasking conditions.

Main Results:

• Inference latency showed a stronger correlation with detection accuracy (mAP) than with FLOPs or parameter count.
• Latency scaling with input size deviated from theoretical predictions due to memory bandwidth limitations.
• Multi-core NPU scheduling offered marginal benefits due to synchronization and memory bottlenecks.
• Memory bandwidth was the primary factor for robustness under multitasking stress.
• Significant energy-per-inference differences were observed across SoCs.

Conclusions:

• Hardware-aware model selection and memory-efficient optimizations are crucial for real-time edge AI applications.
• Performance is dictated by a complex interplay of model architecture, hardware capabilities, and system-level factors, not just TOPS.
• The study provides practical insights for optimizing object detection deployment on embedded platforms.