针对卷积神经网络的通用热散散化
1Mathematics Department, RPTU Kaiserslautern-Landau, Kaiserslautern, 67663, Germany barisin@rptu.de.
Neural computation
|July 24, 2025
概括
本研究介绍了一种数据驱动的修剪方法,用于过度参数化的卷积神经网络 (CNN). 这种新的方法实现了显著的网络稀疏性,精度损失最小,为高效的深度学习模型提供了可扩展的解决方案.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 卷积神经网络 (CNN) 通常过度参数化,导致计算成本昂贵和复杂的架构.
- 鉴定最佳和最小的CNN架构是NP难题,因为有大量可能的网络配置和超参数.
研究的目的:
- 开发一种计算可扩展和数据驱动的方法来修剪过度参数化的CNN.
- 通过高效的架构优化来降低网络复杂性,同时保持高精度.
主要方法:
- 建议采用基于热带放松的逐层修剪技术.
- 该方法利用网络最小化作为预训练CNN的稀疏性约束.
- 采用了一个具有亚线性缩放成本的数值可扩展算法.
主要成果:
- 在基准数据集上实现了高稀疏度水平:在MNIST (LeNet) 上达到55%-84%,在CIFAR-10 (VGG-16,ResNet18) 上达到73%-89%.
- 在实验中保持了最小的准确性损失,范围从0.1%到0.5%.
结论:
- 拟议的修剪方法有效地减少了CNN的复杂性和参数数量.
- 这种数据驱动的方法为创建高效准确的深度学习模型提供了可扩展的解决方案.
相关概念视频
Convolution: Math, Graphics, and Discrete Signals
411
In any LTI (Linear Time-Invariant) system, the convolution of two signals is denoted using a convolution operator, assuming all initial conditions are zero. The convolution integral can be divided into two parts: the zero-input or natural response and the zero-state or forced response, with t0 indicating the initial time.
To simplify the convolution integral, it is assumed that both the input signal and impulse response are zero for negative time values. The graphical convolution process...
To simplify the convolution integral, it is assumed that both the input signal and impulse response are zero for negative time values. The graphical convolution process...
411
Entropy
31.3K
Salt particles that have dissolved in water never spontaneously come back together in solution to reform solid particles. Moreover, a gas that has expanded in a vacuum remains dispersed and never spontaneously reassembles. The unidirectional nature of these phenomena is the result of a thermodynamic state function called entropy (S). Entropy is the measure of the extent to which the energy is dispersed throughout a system, or in other words, it is proportional to the degree of disorder of a...
31.3K
Convolution Properties II
287
The important convolution properties include width, area, differentiation, and integration properties.
The width property indicates that if the durations of input signals are T1 and T2, then the width of the output response equals the sum of both durations, irrespective of the shapes of the two functions. For instance, convolving two rectangular pulses with durations of 2 seconds and 1 second results in a function with a width of 3 seconds.
The area property asserts that the area under the...
The width property indicates that if the durations of input signals are T1 and T2, then the width of the output response equals the sum of both durations, irrespective of the shapes of the two functions. For instance, convolving two rectangular pulses with durations of 2 seconds and 1 second results in a function with a width of 3 seconds.
The area property asserts that the area under the...
287
Convolution Properties I
240
Convolution computations can be simplified by utilizing their inherent properties.
The commutative property reveals that the input and the impulse response of an LTI (Linear Time-Invariant) system can be interchanged without affecting the output:
The commutative property reveals that the input and the impulse response of an LTI (Linear Time-Invariant) system can be interchanged without affecting the output:
240
Entropy and Solvation
7.2K
The process of surrounding a solute with solvent is called solvation. It involves evenly distributing the solute within the solvent. The rule of thumb for determining a solvent for a given compound is that like dissolves like. A good solvent has molecular characteristics similar to those of the compound to be dissolved. For example, polar solutions dissolve polar solutes, and apolar solvents dissolve apolar solutes. A polar solvent is a solvent that has a high dielectric constant (ϵ...
7.2K
Entropy Change in Reversible Processes
2.7K
In the Carnot engine, which achieves the maximum efficiency between two reservoirs of fixed temperatures, the total change in entropy is zero. The observation can be generalized by considering any reversible cyclic process consisting of many Carnot cycles. Thus, it can be stated that the total entropy change of any ideal reversible cycle is zero.
The statement can be further generalized to prove that entropy is a state function. Take a cyclic process between any two points on a p-V diagram.
The statement can be further generalized to prove that entropy is a state function. Take a cyclic process between any two points on a p-V diagram.
2.7K

