通过热带多项式实现神经网络表达能力的尖上限
IEEE transactions on neural networks and learning systems
|February 5, 2024
概括
这项研究得出了一条线性神经网络 (PLNN) 的尖上限,以解决表达力中的指数差距. 新的基于等级和基于精度的方法提高了神经网络分析的准确性和计算效率.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 计算理论 计算理论
背景情况:
- 通过线性区域的数量来衡量神经网络的表达力,由于理论界限之间的指数差距面临挑战.
- 这种差距随着神经网络容量增加而扩大,限制了实际应用和理论理解.
研究的目的:
- 导出一条线性神经网络 (PLNNs) 的表达能力的尖上限.
- 弥合神经网络表达力测量的理论和实践差距.
主要方法:
- 建立了热带多项式和PLNN之间的关系.
- 为未扩展的热带多项式提出了基于等级的方法,超越了扎斯拉夫斯基基于定理的方法.
- 引入了基于扩展热带多项式的精确方法,将指数增长转换为宽度的多项式增长.
主要成果:
- 基于等级的方法有效地减少了交叉的超平面的数量.
- 基于精度的方法证明了线性区域的多项式增长,对于更大的层宽度证明有效.
- 通过经验分析和实验得出并验证了PLNN的利上限.
结论:
- 由此得出的利的上限为PLNN表达力提供了更准确的测量.
- 提出的方法为分析神经网络容量提供了更好的计算效率和准确性.
- 这些发现在模拟数据和现实数据集上得到了验证,证实了它们的实际可行性.
相关概念视频
Ampere-Maxwell's Law: Problem-Solving
630
A parallel-plate capacitor with capacitance C, whose plates have area A and separation distance d, is connected to a resistor R and a battery of voltage V. The current starts to flow at t = 0. What is the displacement current between the capacitor plates at time t? From the properties of the capacitor, what is the corresponding real current?
To solve the problem, we can use the equations from the analysis of an RC circuit and Maxwell's version of Ampère's law.
For the first part of...
To solve the problem, we can use the equations from the analysis of an RC circuit and Maxwell's version of Ampère's law.
For the first part of...
630
Region of Convergence of Laplace Tarnsform
555
The Region of Convergence (ROC) is a fundamental concept in signal processing and system analysis, particularly associated with the Laplace transform. The ROC represents an area in the complex plane where the Laplace transform of a given signal converges, determining the transform's applicability and utility.
Consider a decaying exponential signal that begins at a specific time. When deriving its Laplace transform, the time-domain variable is replaced with a complex variable. This...
Consider a decaying exponential signal that begins at a specific time. When deriving its Laplace transform, the time-domain variable is replaced with a complex variable. This...
555
The Maximum Power Transfer Theorem
620
Consider a linear AC Thevenin equivalent circuit connected to a load impedance.
The load connected draws the current, and the circuit delivers the power to the load. The alternating current flowing through the load is determined using the rectangular form of voltages, currents, network impedance, and load impedance. The average power delivered to the load is obtained from the product of the square of current and load resistance.
The load connected draws the current, and the circuit delivers the power to the load. The alternating current flowing through the load is determined using the rectangular form of voltages, currents, network impedance, and load impedance. The average power delivered to the load is obtained from the product of the square of current and load resistance.
620
Convergence of Fourier Series
148
The Fourier series is a powerful mathematical tool for representing periodic signals as an infinite sum of complex exponentials. In practice, this infinite series is truncated to a finite number of terms, yielding a partial sum. This truncation makes the approximation of the signal feasible but introduces certain challenges, particularly near discontinuities, known as the Gibbs phenomenon.
The Gibbs phenomenon refers to the persistent oscillations and overshoots that occur near discontinuities...
The Gibbs phenomenon refers to the persistent oscillations and overshoots that occur near discontinuities...
148
Parseval's Theorem
510
Parseval's theorem is a fundamental concept in signal processing and harmonic analysis. It asserts that for a periodic function, the average power of the signal over one period equals the sum of the squared magnitudes of all its complex Fourier coefficients. This theorem, named after Marc-Antoine Parseval, provides a powerful tool for analyzing the energy distribution in signals.
Interestingly, Parseval's theorem also holds for the trigonometric form of the Fourier series, which...
Interestingly, Parseval's theorem also holds for the trigonometric form of the Fourier series, which...
510
Norton's Theorem
594
Norton's theorem is a fundamental principle stating that a linear two-terminal circuit can be substituted with an equivalent circuit, which comprises a current source (ⅠN) in parallel with a resistor (RN). Here, ⅠN represents the short-circuit current flowing through the terminals, and RN stands for the input or equivalent resistance at the terminals when all independent sources are deactivated. This implies that the circuit illustrated in Figure (a) can be exchanged with the...
594


