贝叶斯有效动作的预测能力对于完全连接的隐藏层神经网络在比例极限中的预测能力
P Baglioni1,2, R Pacelli3,4, R Aiudi1,2
1Dipartimento di Scienze Matematiche, Fisiche e Informatiche, <a href="https://ror.org/02k7wn190">Università degli Studi di Parma</a>, Parco Area delle Scienze, 7/A 43124 Parma, Italy.
Physical review letters
|July 29, 2024
概括
这项研究使用数值实验验验证了贝叶斯对浅层神经网络的有效作用. 结果显示出极好的一致性,表明全球调整规模是这些网络内核重规范化的关键.
科学领域:
- 机器学习 机器学习
- 统计物理 统计物理
- 深度学习理论 深度学习理论
背景情况:
- 贝叶斯有效行动理论为理解神经网络行为提供了一个框架.
- 浅层神经网络架构是深度学习的基础.
- 了解概括错误对于模型性能至关重要.
研究的目的:
- 经验验证一个新的贝叶斯有效作用对于浅层神经网络.
- 确定该理论准确预测网络行为的条件.
- 调查温度和先前大小等参数对概括错误的作用.
主要方法:
- 使用完全连接的一个隐藏层神经网络进行数值实验.
- 关于MNIST和CIFAR10数据集的培训网络,具有离散的朗格文动态.
- 将实验概括错误与贝叶斯有效动作的预测进行比较.
主要成果:
- 精确的数值实验表明,有效理论与实证结果之间存在很强的一致性.
- 通过各种参数 (温度,先验,层大小,数据集大小) 探索理论的预测能力.
- 无限宽度内核的全球重新缩放被确定为内核重新规范化的重要机制.
结论:
- 贝叶斯有效作用是比例极限中浅层神经网络的有效和预测理论.
- 全球重新缩放在标准缩放贝叶斯浅层网络的内核重规范化中发挥着关键作用.
- 这项工作将统计物理学和深度学习理论结合起来,为网络概括提供了洞察力.
相关概念视频
PI Controller: Design
238
Proportional Integral (PI) controllers are a fundamental component in modern control systems, widely used to enhance performance and mitigate steady-state errors. They are particularly effective in applications such as automatic brightness adjustment on smartphones, where they excel at mitigating steady-state errors for step-function inputs. Unlike PD controllers, which require time-varying errors to function optimally, PI controllers leverage their integral component to address residual...
238
Action Potentials
130.5K
Overview
130.5K
Action Potential
7.9K
Neurons communicate by firing action potentials—the electrochemical signal that is propagated along the axon. The signal results in the release of neurotransmitters at axon terminals, thereby transmitting information to the nervous system. An action potential is a specific "all-or-none" change in membrane potential that results in a rapid spike in voltage.
Membrane potential in neurons
Neurons typically have a resting membrane potential of about -70 millivolts (mV). When they...
Membrane potential in neurons
Neurons typically have a resting membrane potential of about -70 millivolts (mV). When they...
7.9K
PD Controller: Design
210
In automotive engineering, car suspension systems often employ Proportional Derivative (PD) controllers to enhance performance. PD controllers are utilized to adjust the damping force in response to road conditions. A controller, acting as an amplifier with a constant gain, demonstrates proportional control, with output directly mirroring input.
Designing a continuous-data controller requires selecting and linking components like adders and integrators, which are fundamental in Proportional,...
Designing a continuous-data controller requires selecting and linking components like adders and integrators, which are fundamental in Proportional,...
210
Time and frequency -Domain Interpretation of PI Control
113
Proportional-Integral (PI) controllers are essential in many control systems to improve stability and performance. They are commonly used in everyday devices like thermostats to enhance system damping and reduce steady-state error. When the zero in the controller's transfer function is optimally placed, the system benefits significantly in terms of stability and accuracy.
Acting as a low-pass filter, the PI controller slows the system's response and extends settling times. This requires...
Acting as a low-pass filter, the PI controller slows the system's response and extends settling times. This requires...
113
Linear Approximation in Frequency Domain
88
Linear systems are characterized by two main properties: superposition and homogeneity. Superposition allows the response to multiple inputs to be the sum of the responses to each individual input. Homogeneity ensures that scaling an input by a scalar results in the response being scaled by the same scalar.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
88


