ShaderNN:一种轻量级且高效的推理引擎,用于移动GPU上的实时应用
Jing Xie1,2, Yuzhong Yan2, Abhishek Saxena2
1Department of Electrical and Computer Engineering, University of Maryland at College Park, 8223 Paint Branch Dr, College Park, MD, 20740, USA.
概括
影子神经网络 (ShaderNN) 是一个新的基于OpenGL的框架,用于在移动设备上进行高效的深度学习推断. 它最大限度地减少了数据的移动,并提高了性能,超过了像TensorFlow-Lite这样的现有解决方案.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 移动计算 移动计算
背景情况:
- 在移动设备上深度神经网络推断面临由于有限资源 (计算,功率,内存) 的挑战.
- 实时应用程序需要最小化数据移动和增加数据局部性,以实现高效的推理.
- 现有的推理引擎通常涉及CPU和GPU之间昂贵的数据传输.
研究的目的:
- 提出Shader神经网络 (ShaderNN),这是一个基于OpenGL的移动设备的快速和高效的推断框架.
- 在移动深度学习中应对有限的计算能力,电力预算和数据流动的挑战.
- 为了实现与实时图形和图像处理应用程序的无集成.
主要方法:
- 使用OpenGL开发了ShaderNN,利用基于纹理的输入/输出来实现零复制集成.
- 用于神经网络推理运算符的片段遮蔽器,特别是用于较小的模型.
- 实现了混合计算和碎片遮光器方法,用于层级遮光器选择,以优化性能.
- 采用了OpenGL功能,如规范化,插值和纹理填充,以提高性能.
主要成果:
- 与TensorFlow-Lite相比,ShaderNN在使用高通和MediaTek芯片的移动设备上表现出更高的性能.
- 该框架通过最大限度地减少CPU和GPU之间的数据传输来实现高效,节能推断.
- 一个案例研究证实了ShaderNN在Android媒体处理应用程序中的可用性和无集成.
结论:
- ShaderNN提供了一种新且有效的解决方案,用于在设备上进行深度学习推断,并优化移动约束.
- 基于纹理的,以OpenGL为中心的方法提供了显著的性能和效率提升.
- ShaderNN是将深度学习集成到移动应用程序中的可行和高性能替代方案.
相关概念视频
Parallel Processing
145
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
145
Accelerating Fluids
1.0K
When a fluid is in constant acceleration, the pressure and buoyant force equations are modified. Suppose a beaker is placed in an elevator accelerating upward with a constant acceleration, a. In the beaker, assume there is a thin cylinder of height h with an infinitesimal cross-sectional area, ΔS.
The motion of the liquid within this infinitesimal cylinder is considered to obtain the pressure difference. Three vertical forces act on this liquid:
The motion of the liquid within this infinitesimal cylinder is considered to obtain the pressure difference. Three vertical forces act on this liquid:
1.0K
Singularity Functions for Shear
120
In structural analysis, singularity functions are crucial in simplifying the representation of shear forces in beams under discontinuous loading. These functions describe discontinuous variations in shear force across a beam with varying loads by using a single mathematical expression, regardless of the complexity of the loading conditions. The singularity functions are derived from creating a free-body diagram of the beam and then making conceptual cuts at specific points to examine the...
120
Statically Indeterminate Problem Solving
360
Statically indeterminate problems are those where statics alone can not determine the internal forces or reactions. Consider a structure comprising two cylindrical rods made of steel and brass. These rods are joined at point B and restrained by rigid supports at points A and C. Now, the reactions at points A and C and the deflection at point B are to be determined. This rod structure is classified as statically indeterminate as the structure has more supports than are necessary for maintaining...
360
Navier–Stokes Equations
422
For incompressible Newtonian fluids, where density remains constant, stresses show a linear relationship with the deformation rate, defined by normal and shear stresses. Normal stresses depend on the pressure exerted on the fluid and the rate of deformation in specific directions, which determines how fluid flows under varying pressures. Shear stresses, on the other hand, act tangentially across fluid layers. They explain how adjacent fluid layers slide relative to one another, connecting...
422
Maxwell-Boltzmann Distribution: Problem Solving
1.4K
Individual molecules in a gas move in random directions, but a gas containing numerous molecules has a predictable distribution of molecular speeds, which is known as the Maxwell-Boltzmann distribution, f(v).
This distribution function f(v) is defined by saying that the expected number N (v1,v2) of particles with speeds between v1 and v2 is given by
This distribution function f(v) is defined by saying that the expected number N (v1,v2) of particles with speeds between v1 and v2 is given by
1.4K


