在残余网络中实现最佳信号传播的场理论
Kirsten Fischer1,2, David Dahmen1, Moritz Helias1,3
1Jülich Research Centre, Jülich, Institute for Advanced Simulation (IAS-6), Germany.
Physical review. E
|January 21, 2026
概括
具有跳过连接的剩余网络 (ResNets) 提供更好的性能. 本研究开发了一个理论,解释了ResNets中的缩放参数如何提高性能,并在超参数中概括.
科学领域:
- 深度学习理论 深度学习理论
- 神经网络架构 神经网络架构
- 机器学习 机器学习
背景情况:
- 由于跳过连接,剩余网络 (ResNets) 在深层架构中优于前网络.
- 剩余分支中的缩放参数增强了概括性,但其机制和普遍性尚不清楚.
研究的目的:
- 为剩余网络开发一个有限大小的场理论.
- 分析信号传播和残余分支扩展参数的影响.
- 解释扩展参数的性能改进和普遍性.
主要方法:
- 对ResNets的系统有限大小场理论的推导.
- 响应函数的分析推导,以测量输入灵敏度.
- 对缩放参数对网络超参数的依赖性的分析.
主要成果:
- 经验上最优的缩放参数值位于深度ResNets的最大网络灵敏度范围之内.
- 获得了最佳缩放参数的分析表达式,显示了对重量差等超参数的微弱依赖.
- 这就解释了观察到的缩放参数效率的普遍性.
结论:
- 建立了一个分析有限大小ResNets的理论框架.
- 这项研究提供了对残余分支扩展参数的好处的机制性理解.
- 这些发现为提高性能提供了对超参数调整和网络设计的见解.
更多相关视频
05:30Soft Pneumatic Robot Modulates Graph Theory Metrics of Brain Network for Hand Rehabilitation After Stroke
Published on: October 10, 2025
440
11:27Interfacing Microfluidics with Microelectrode Arrays for Studying Neuronal Communication and Axonal Signal Propagation
Published on: December 8, 2018
8.5K
相关概念视频
Crystal Field Theory - Octahedral Complexes
30.7K
Crystal Field Theory
To explain the observed behavior of transition metal complexes (such as colors), a model involving electrostatic interactions between the electrons from the ligands and the electrons in the unhybridized d orbitals of the central metal atom has been developed. This electrostatic model is crystal field theory (CFT). It helps to understand, interpret, and predict the colors, magnetic behavior, and some structures of coordination compounds of transition metals.
CFT focuses on...
To explain the observed behavior of transition metal complexes (such as colors), a model involving electrostatic interactions between the electrons from the ligands and the electrons in the unhybridized d orbitals of the central metal atom has been developed. This electrostatic model is crystal field theory (CFT). It helps to understand, interpret, and predict the colors, magnetic behavior, and some structures of coordination compounds of transition metals.
CFT focuses on...
30.7K
Optimal Arousal Theory
788
The optimal arousal theory suggests that performance is maximized when an individual experiences a moderate level of arousal. This theory is closely tied to the Yerkes-Dodson law, which illustrates an inverted U-shaped relationship between arousal and performance. The law, formulated by psychologists Robert Yerkes and John Dodson, implies an ideal arousal level for optimal performance, and deviations from this level can lead to declines in effectiveness.
Inverted U-Shaped Performance Curve
The...
Inverted U-Shaped Performance Curve
The...
788
Crystal Field Theory - Tetrahedral and Square Planar Complexes
48.3K
Tetrahedral Complexes
Crystal field theory (CFT) is applicable to molecules in geometries other than octahedral. In octahedral complexes, the lobes of the dx2−y2 and dz2 orbitals point directly at the ligands. For tetrahedral complexes, the d orbitals remain in place, but with only four ligands located between the axes. None of the orbitals points directly at the tetrahedral ligands. However, the dx2−y2 and dz2 orbitals (along the Cartesian axes) overlap with the ligands less than the dxy,...
Crystal field theory (CFT) is applicable to molecules in geometries other than octahedral. In octahedral complexes, the lobes of the dx2−y2 and dz2 orbitals point directly at the ligands. For tetrahedral complexes, the d orbitals remain in place, but with only four ligands located between the axes. None of the orbitals points directly at the tetrahedral ligands. However, the dx2−y2 and dz2 orbitals (along the Cartesian axes) overlap with the ligands less than the dxy,...
48.3K
Residual Stresses
599
Residual stresses reside in a structure even after removing the original stress inducer. This phenomenon often arises from varied plastic deformations across different parts of a structure. Consider a rod stretched beyond its yield point. It will not regain its original length due to permanent deformation. Even after load removal, the rod does not entirely lose stress because of uneven plastic deformations, resulting in residual stresses. The computation of these stresses in structures is...
599
Residual Plots
6.2K
A residual plot is a statistical representation of data used to analyze correlation and regression results. It helps verify the requirements for drawing specific conclusions about correlation and regression. To obtain the residual plot, first, the residual for each data value is calculated, which is simply the vertical distance between the observed and the predicted value obtained from the regression equation.
When the residual values are plotted against the variable x, it is called a residual...
When the residual values are plotted against the variable x, it is called a residual...
6.2K
Band Theory
17.1K
When two or more atoms come together to form a molecule, their atomic orbitals combine and molecular orbitals of distinct energies result. In a solid, there are a large number of atoms, and therefore a large number of atomic orbitals that may be combined into molecular orbitals. These groups of molecular orbitals are so closely placed together to form continuous regions of energies, known as the bands.
The energy difference between these bands is known as the band gap.
Conductor, Semiconductor,...
The energy difference between these bands is known as the band gap.
Conductor, Semiconductor,...
17.1K
