关于正方形网络的表达力和可训练性
IEEE transactions on neural networks and learning systems
|November 23, 2023
概括
与传统网络相比,二进制人工神经元提供了优越的模型表达性. 一个新的培训策略,引用线性初始化 (ReLinear),稳定二级网络,以提高深度学习性能.
科学领域:
- 人工智能的人工智能是人工智能.
- 深度学习是一种深度学习.
- 计算神经科学是一种计算神经科学.
背景情况:
- 生物神经元启发了二次数的人工神经元,它用二次数函数取代了内部产品运算.
- 现有的二次神经网络看起来很有前途,但缺乏理论基础,面临训练不稳定.
研究的目的:
- 从理论上证明二次神经网络的优越表达性.
- 开发一个稳定的培训策略,用于二级神经网络.
主要方法:
- 应用了线条理论和代数几何学来证明增强的模型表达性.
- 引入了参考线性初始化 (ReLinear) 来稳定二级网络训练.
主要成果:
- 两个定理证实二级网络比传统网络具有更大的模型表达性.
- "再线性"战略有效地稳定了培训,减轻了崩风险.
- 实验验证了二级深度学习模型的增强性能.
结论:
- 四位数神经网络在模型表达性方面提供了显著的优势.
- 该ReLinear培训策略释放了二次式深度学习的全部潜力.
相关概念视频
Multi-input and Multi-variable systems
108
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
108
State Space to Transfer Function
212
The conversion of state-space representation to a transfer function is a fundamental process in system analysis. It provides a method for transitioning from a time-domain description to a frequency-domain representation, which is crucial for simplifying the analysis and design of control systems.
The transformation process begins with the state-space representation, characterized by the state equation and the output equation. These equations are typically represented as:
The transformation process begins with the state-space representation, characterized by the state equation and the output equation. These equations are typically represented as:
212
Network Function of a Circuit
294
Frequency response analysis in electrical circuits provides vital insights into a circuit's behavior as the frequency of the input signal changes. The transfer function, a mathematical tool, is instrumental in understanding this behavior. It defines the relationship between phasor output and input and comes in four types: voltage gain, current gain, transfer impedance, and transfer admittance. The critical components of the transfer function are the poles and zeros.
294
Transfer Function to State Space
266
State-space representation is a powerful tool for simulating physical systems on digital computers, necessitating the conversion of the transfer function into state-space form. Consider an nth-order linear differential equation with constant coefficients, like those encountered in an RLC circuit. The state variables are selected as the output and its n−1 derivatives. Differentiating these variables and substituting them back into the original equation produces the state equations.
In an...
In an...
266
Transfer Function in Control Systems
508
The transfer function is a fundamental concept in the analysis and design of linear time-invariant (LTI) systems. It offers a concise way to understand how a system responds to different inputs in the frequency domain. It serves as a bridge between the time-domain differential equations that describe system dynamics and the frequency-domain representation that facilitates easier manipulation and analysis.
To derive the transfer function, consider a general nth-order linear time-invariant...
To derive the transfer function, consider a general nth-order linear time-invariant...
508
Linear Approximation in Frequency Domain
93
Linear systems are characterized by two main properties: superposition and homogeneity. Superposition allows the response to multiple inputs to be the sum of the responses to each individual input. Homogeneity ensures that scaling an input by a scalar results in the response being scaled by the same scalar.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
93


