科伦:同时推断和持续训练在成本效益高的基于人工智能的移动图像传感边缘
Yu Liu1, Anurag Andhare1, Kyoung-Don Kang1
1Department of Computer Science, State University of New York at Binghamton, 4400 Vestal Parkway East, Binghamton, NY 13902, USA.
Sensors (Basel, Switzerland)
|August 29, 2024
概括
Corun是一个新的框架,可以在单个GPU上高效处理多个深度学习图像分析任务和模型再训练. 这种方法可以提高推断速度,并保持移动应用程序的准确性,同时降低成本.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 边缘计算 边缘计算
背景情况:
- 基于深度学习的移动图像传感面临着设备资源有限的挑战,导致高延迟和电池耗尽.
- 推理准确性可能随着时间的推移而降低,由于数据漂移,需要更新模型.
- 目前的边缘部署通常需要单独的GPU来推断和重新训练,增加成本.
研究的目的:
- 推出Corun,一个成本高效的框架,用于在单边GPU上同时进行深度学习推断和模型再训练.
- 为了提高推断吞吐量并保持移动图像传感应用的准确性.
- 为了降低与边缘AI部署相关的硬件成本.
主要方法:
- 科伦利用离线分析来确定可以在一个GPU上进行再训练的最佳数量的并发推理查询.
- 它采用调度方法来管理多个推断任务和持续的模型重新训练,而不会导致内存外错误或显著的延迟增加.
- 该框架专注于边缘服务器中的单个商品GPU.
主要成果:
- 科伦显著提高了推断吞吐量,随着并发推断查询数量的增加而扩展.
- 与单独任务处理相比,延迟和重新训练时代长度的增长率要低得多.
- 评估证实了Coron在降低GPU需求和部署成本方面的成本效益.
结论:
- 对于资源有限的边缘环境,Corun提供了一个实用的解决方案,实现了基于深度学习的高效移动图像传感.
- 通过整合推断和再培训,Corun展示了一种可行的战略,用于降低边缘AI的成本.
- 该框架成功地解决了移动深度学习应用中的延迟,电池消耗和精度维护挑战.
相关概念视频
01:20Parallel Processing
147
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
147
02:07Improving Translational Accuracy
9.4K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
9.4K
01:30Machines: Problem Solving II
303
Machines are complex structures consisting of movable, pin-connected multi-force members that work together to transmit forces. Consider a lifting tong carrying a 100 kg load. It comprises movable sections DAF and CBG linked together with member AB.
303
01:22Multi-input and Multi-variable systems
105
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
105
01:22Force Classification
1.2K
Forces play a crucial role in the study of physics and engineering. They are essential in describing the motion, behavior, and equilibrium of objects in the physical world. Forces can be classified based on their origin, type, and direction of action.
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
1.2K
01:24Convolution: Math, Graphics, and Discrete Signals
236
In any LTI (Linear Time-Invariant) system, the convolution of two signals is denoted using a convolution operator, assuming all initial conditions are zero. The convolution integral can be divided into two parts: the zero-input or natural response and the zero-state or forced response, with t0 indicating the initial time.
To simplify the convolution integral, it is assumed that both the input signal and impulse response are zero for negative time values. The graphical convolution process...
To simplify the convolution integral, it is assumed that both the input signal and impulse response are zero for negative time values. The graphical convolution process...
236