硬件辅助低延迟NPU虚拟化方法用于多传感器AI系统
Jong-Hwan Jean1, Dong-Sun Kim1
1Department of Semiconductor Systems Engineering, Sejong University, Seoul 05006, Republic of Korea.
Sensors (Basel, Switzerland)
|January 8, 2025
概括
这项研究通过虚拟化神经处理单元 (NPU) 来增强AI处理,同时运行多个模型. 这提高了资源利用率,并减少了实时应用程序 (如自动驾驶) 的延迟.
科学领域:
- 人工智能的人工智能
- 计算机工程 计算机工程
- 嵌入式系统 嵌入式系统
背景情况:
- 自动驾驶和智能家居中的AI系统需要处理复杂的文本和图像数据.
- 当前的多传感器系统面临着低资源利用率和内存延迟的挑战.
- 整合NPU和传感器可以提高速度,但并不总是提高效率.
研究的目的:
- 减少人工智能系统的处理时间和提高资源利用率.
- 为了应对基于NPU的系统中低资源利用和内存延迟的挑战.
- 在资源有限的环境中实现高效的多任务处理和低延迟处理.
主要方法:
- 虚拟化神经处理单元 (NPU) 以同时处理多个深度学习模型.
- 实施硬件调度器来管理任务执行和优化资源分配.
- 使用数据预检查技术来最大限度地减少内存延迟.
主要成果:
- 硬件调度器在测试模型中减少了超过10%的内存周期.
- 观察到记忆延迟的显著减少,例如,NCF的30%和DLRM的70%.
- 尽量减少NPU空时间和内存延迟,特别是在频繁上下文切换的环境中.
结论:
- 拟议的虚拟化方法有效地提高了NPU资源利用率,并减少了处理延迟.
- 这种方法对于需要高效的多任务处理的实时人工智能应用非常有益.
- 在资源有限的环境中实现了优化性能,支持自动驾驶和智能家居等应用程序.
相关概念视频
01:22Multi-input and Multi-variable systems
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence of...
In the absence of...
01:20Parallel Processing
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...