走向高效的卷积神经网络与结构化的三元模式
IEEE transactions on neural networks and learning systems
|April 23, 2024
概括
结构化三元模式 (STePs) 通过使用静态过器而不是可学习权重来创建高效的深度学习模型. 这种方法可以减少移动和嵌入式应用程序的资源需求,提高性能并降低可训练参数.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 深度学习 (DL) 模型,特别是卷积神经网络 (ConvNets),需要大量的计算资源,限制它们在资源有限的设备上部署.
- 高效的DL模型对于在设备上的应用和降低培训成本至关重要.
- 目前的ConvNet架构通常会给移动和嵌入式平台带来挑战,因为它们对资源的需求很高.
研究的目的:
- 通过利用来自本地二进制模式 (LBPs) 和 Haar 特性的静态卷积过器来开发高效的 ConvNet 架构.
- 引入结构化三进制模式 (STePs) 作为在网络初始化过程中生成不可学习过器的方法.
- 为了减少DL模型中可训练参数和内存足迹的数量.
主要方法:
- 从LBP和Haar特征生成静态卷积波器,称为结构化三角形模式 (STePs).
- 启动的ConvNet架构使用STeP,而不是可学习的权重参数.
- 评估了关于四个图像分类数据集和基于无人机 (UAV) 的空中车辆检测的拟议方法.
主要成果:
- 在保持高检测精度的同时,STeP显著减少了40%-80%的可训练参数.
- 与STeP集成的共同网络骨干显示了提高效率和竞争性分类结果.
- 基于STeP的定制网络为设备上的应用程序提供了有利的权衡.
结论:
- 结构化三元模式 (STePs) 提供了一个有效的方法来创建高效的DL架构.
- 使用不可学习的预先生成过器可以大幅降低模型的复杂性和资源需求.
- 这项研究鼓励进一步探索以先前为基础的非可学习权重,以提高DL模型的效率,而无需在培训后进行修改.
相关概念视频
01:25Neural Circuits
1.2K
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
1.2K
01:24Convolution: Math, Graphics, and Discrete Signals
249
In any LTI (Linear Time-Invariant) system, the convolution of two signals is denoted using a convolution operator, assuming all initial conditions are zero. The convolution integral can be divided into two parts: the zero-input or natural response and the zero-state or forced response, with t0 indicating the initial time.
To simplify the convolution integral, it is assumed that both the input signal and impulse response are zero for negative time values. The graphical convolution process...
To simplify the convolution integral, it is assumed that both the input signal and impulse response are zero for negative time values. The graphical convolution process...
249
01:17Convolution Properties II
192
The important convolution properties include width, area, differentiation, and integration properties.
The width property indicates that if the durations of input signals are T1 and T2, then the width of the output response equals the sum of both durations, irrespective of the shapes of the two functions. For instance, convolving two rectangular pulses with durations of 2 seconds and 1 second results in a function with a width of 3 seconds.
The area property asserts that the area under the...
The width property indicates that if the durations of input signals are T1 and T2, then the width of the output response equals the sum of both durations, irrespective of the shapes of the two functions. For instance, convolving two rectangular pulses with durations of 2 seconds and 1 second results in a function with a width of 3 seconds.
The area property asserts that the area under the...
192
01:20Convolution Properties I
147
Convolution computations can be simplified by utilizing their inherent properties.
The commutative property reveals that the input and the impulse response of an LTI (Linear Time-Invariant) system can be interchanged without affecting the output:
The commutative property reveals that the input and the impulse response of an LTI (Linear Time-Invariant) system can be interchanged without affecting the output:
147