规模适应性近同系不变网络的高效学习
Zhengyang Shen1, Yeqing Qiu2, Jialun Liu1
1Baidu Inc, Beijing, 100871, China.
概括
本研究介绍了一种有效的方法,通过将其分解为子组,将相亲不变性纳入神经网络. 这种方法显著降低了计算成本,同时在分类任务中获得了最先进的结果.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 将不变性纳入神经网络对于强大的性能至关重要.
- 由于对大型转换组进行直接采样,对亲属不变的现有方法在计算上昂贵.
研究的目的:
- 开发一种计算效率高的方法,在神经网络中实现近乎精确的亲系不变性.
- 为了解决需要广泛采样的现有方法的局限性.
主要方法:
- 将亲属不变分解为欧几里德群E (n) 和单轴缩放群US (n).
- 采用一个E (n) -invariant模型用于E (n) -invariance和数据增强用于US (n) -invariance.
- 在训练期间实施自适应尺度增大,以防止过度的尺度不变.
主要成果:
- 与现有的方法相比,实现了显著较低的计算复杂性 (2D中的O(N^2),3D中的O(N^4) 与现有的方法相比 (2D中的O(N^6),3D中的O(N^12).
- 在affNIST和SIM2MNIST分类任务中获得了新的最新结果.
- 将推断时间缩短到不到15%,并减少计算资源和模型参数.
结论:
- 拟议的子组分解方法有效地将亲属不变性纳入神经网络.
- 这种新的方法提供了一种实用且优越的替代方案,可以替代现有的计算密集型方法来实现相似不变性.
- 该方法表现出强大的性能和效率,为亲属不变神经网络设定了新的标准.
相关概念视频
01:26Linear Approximation in Frequency Domain
89
Linear systems are characterized by two main properties: superposition and homogeneity. Superposition allows the response to multiple inputs to be the sum of the responses to each individual input. Homogeneity ensures that scaling an input by a scalar results in the response being scaled by the same scalar.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
89
01:23Linear time-invariant Systems
257
A system is linear if it displays the characteristics of homogeneity and additivity, together termed the superposition property. This principle is fundamental in all linear systems. Linear time-invariant (LTI) systems include systems with linear elements and constant parameters.
The input-output behavior of an LTI system can be fully defined by its response to an impulsive excitation at its input. Once this impulse response is known, the system's reaction to any other input can be...
The input-output behavior of an LTI system can be fully defined by its response to an impulsive excitation at its input. Once this impulse response is known, the system's reaction to any other input can be...
257
01:26Scaling
245
In designing and analyzing filters, resonant circuits, or circuit analysis at large, working with standard element values like 1 ohm, 1 henry, or 1 farad can be convenient before scaling these values to more realistic figures. This approach is widely utilized by not employing realistic element values in numerous examples and problems; it simplifies mastering circuit analysis through convenient component values. The complexity of calculations is thereby reduced, with the understanding that...
245