在初始化时修剪 - 一个草图视角
IEEE transactions on pattern analysis and machine intelligence
|August 18, 2025
概括
彩票假设 (LTH) 揭示了在初始化时修剪神经网络类似于矩阵素描. 这种连接提供了新的见解,并改进了修剪算法,特别是当数据独立性是关键时.
科学领域:
- 机器学习 机器学习
- 深度学习理论 深度学习理论
- 神经网络优化神经网络优化
背景情况:
- 彩票假设 (LTH) 提出,密集的神经网络包含较小的子网络,这些子网络在隔离训练时可以达到类似的准确性.
- 在初始化时修剪神经网络是有效模型设计的关键研究领域.
研究的目的:
- 在线设置中分析彩票假设 (LTH).
- 建立LTH与有效矩阵乘法素描问题的理论联系.
- 开发改进的算法,用于初始化时修剪神经网络.
主要方法:
- 在线模型设置中研究彩票假设 (LTH).
- 在初始化和矩阵素描问题中找到稀疏面具之间建立等价性.
- 通过初始化面具限制了经过修剪的线性模型的近似误差.
- 分析稀疏网络搜索的数据独立性.
主要成果:
- 在初始化时找到稀疏的面具相当于矩阵素描.
- 对稀疏网络的数据独立搜索的理论理由.
- 在初始化时对现有的修剪算法进行了全新的,通用的改进.
- 在数据独立场景中提议的改进的证明好处.
结论:
- 草图视角为理解和分析彩票假设 (LTH) 提供了一个强大的框架.
- 拟议的算法改进提高了修剪效率,特别是在数据独立的设置中.
- 这项工作将矩阵素描的理论见解与实际的神经网络修剪策略相结合.
相关概念视频
Drawing Free-body Diagrams: Rules
13.3K
The first step in describing and analyzing most phenomena in physics involves the careful drawing of a free-body diagram. Free-body diagrams are useful in analyzing forces acting on an object or system, and are employed extensively in the study and application of Newton's laws of motion. The steps to draw a free-body diagram are listed below:
13.3K
Vector Algebra: Graphical Method
13.5K
Vectors can be multiplied by scalars, added to other vectors, or subtracted from other vectors. The vector sum of two (or more) vectors is called the resultant vector or, for short, the resultant.
We use the laws of geometry to construct resultant vectors, followed by trigonometry to find vector magnitudes and directions. For a geometric construction of the sum of two vectors in a plane, we follow the parallelogram rule. Suppose two vectors are at arbitrary positions. Translate either one of...
We use the laws of geometry to construct resultant vectors, followed by trigonometry to find vector magnitudes and directions. For a geometric construction of the sum of two vectors in a plane, we follow the parallelogram rule. Suppose two vectors are at arbitrary positions. Translate either one of...
13.5K
Concepts and Prototypes
223
The human nervous system handles vast amounts of information by translating sensory stimuli into neural impulses, which the brain processes, creating thoughts expressed through language or stored as memories. The brain also synthesizes information from emotions and memories, which significantly influence thoughts and behaviors. This intricate process creates a comprehensive mental picture.
The brain organizes this information using concepts, which are mental categories grouping linguistic data,...
The brain organizes this information using concepts, which are mental categories grouping linguistic data,...
223
Simple Trusses
2.0K
A truss is a structural framework consisting of slender members connected at joints, designed to support external loads while minimizing material usage and weight. Simple trusses are a type of planar truss where all members lie within a single two-dimensional plane.
The most basic planar truss is a simple truss with three members arranged in a triangular formation. This triangular truss is inherently stable and rigid due to its geometry, making it an ideal starting point for creating more...
The most basic planar truss is a simple truss with three members arranged in a triangular formation. This triangular truss is inherently stable and rigid due to its geometry, making it an ideal starting point for creating more...
2.0K
Introduction to Vertical Curves
158
Vertical curves are parabolic transitions that connect different grades on highways and railroads, ensuring a smooth alignment between back and forward tangents. The back tangent represents the initial grade, while the forward tangent defines the subsequent grade. These curves can be symmetrical, with equal tangent lengths, or nonsymmetrical, with varying lengths. The key points defining a vertical curve include the Point of Vertical Intersection (P.V.I.), where the tangents meet; the Point of...
158
Boundary Conditions: Lossless Lines
152
Consider a single-phase, two-wire, lossless transmission line terminated by an impedance at the receiving end and a source with Thevenin voltage and impedance at the sending end. The line, with length, has a surge impedance and wave velocity determined by the line's inductance and capacitance.
At the receiving end, the boundary condition states that the voltage equals the product of the receiving-end impedance and current. This relationship is expressed as a function of the incident and...
At the receiving end, the boundary condition states that the voltage equals the product of the receiving-end impedance and current. This relationship is expressed as a function of the incident and...
152


