双的轻量级战略 双的轻量级战略
IEEE transactions on pattern analysis and machine intelligence
|October 10, 2024
概括
这项研究介绍了TEDEPR,这是一种新的双粒轻量化策略,可以在极端稀疏的情况下提高深度神经网络的性能. TEDEPR通过使用张量理论在初始化时进行修剪来提高准确性和效率.
科学领域:
- 深度学习模型优化优化
- 用于边缘计算的人工智能
- 神经网络中的计算效率
背景情况:
- 模型压缩技术,如初始化时的修剪,对于在资源受限的边缘设备上部署深度神经网络至关重要.
- 当前的修剪方法在极端稀疏水平下与性能退化作斗争.
- 简化深度神经网络可以减少存储,加快训练和推理,降低能源消耗.
研究的目的:
- 在极端稀疏条件下提高深度神经网络模型的性能.
- 为推出一种名为TEDEPR的新型双粒轻量级策略.
- 在模型训练之前利用张量理论来优化稀疏的子网络结构.
主要方法:
- TEDEPR采用了一种双粒度的方法:粗粒度的低级张量分解和细粒度的重量修剪.
- 粗粒度水平:重量矩阵/压缩机被分解成低级形式,使用多步链式操作来增强特征提取.
- 细粒度级别:在训练之前,在低级别模型中,不重要的重量根据训练能力进行修剪.
主要成果:
- 在各种数据集 (MNIST,CIFAR,ImageNet) 和架构 (LeNet,ResNet,Transformer) 上进行了广泛的实验,证明了TEDEPR的有效性.
- 在极端稀疏的情况下,TEDEPR在初始化方法上的修剪比现有的修剪方法更准确.
- 该方法导致更快的训练和推理时间,并减少了模型存储空间.
结论:
- 在初始化时,TEDEPR提供了一种优越的修剪方法,特别是在极端稀疏的情况下实现高性能.
- 张量理论的整合为优化稀疏子网络结构提供了一个新的机制.
- TEDEPR促进了在边缘设备上部署高性能模型,推进智能系统.
相关概念视频
Design Example: Managing Concrete Workability
74
This example deals with managing the workability of concrete for a raft foundation project under hot weather conditions. Workability is crucial for ensuring the concrete is easy to place, compact, and finish. In this scenario, a slump test — a common method to measure the workability of fresh concrete — initially indicated low workability. This was attributed to the rapid water loss from the concrete mix, exacerbated by the high temperatures causing the course aggregates to heat up.
74
Design Example: Aggregate Gradation
91
The right type and quality of aggregates are crucial for concrete as they significantly influence its properties, mix proportions, and cost-effectiveness. If different sources are available for sand, the commonly used fine aggregate in concrete, the selection of sand is primarily based on its gradation.
The grading, or particle-size distribution, of sand is determined using sieve analysis, with standard sizes ranging from 150 μm to 10 mm (ASTM No. 100 sieve to 3⁄8 in. sieve). Sand is...
The grading, or particle-size distribution, of sand is determined using sieve analysis, with standard sizes ranging from 150 μm to 10 mm (ASTM No. 100 sieve to 3⁄8 in. sieve). Sand is...
91
Design Example: Maintaining Level of an Embankment
52
Constructing a roadway embankment over uneven terrain requires precise leveling to ensure stability and proper drainage. Surveyors use a leveling instrument and staff to calculate ground elevations and determine the required fill material at each point along the embankment alignment.The process begins by positioning a leveling instrument near a benchmark with a known elevation. A backsight reading establishes the instrument height, which serves as a reference for subsequent measurements. A...
52
Types of Aggregate Grading
428
Aggregate grading is crucial in economically obtaining a concrete mix with adequate strength, reasonable workability, and minimal segregation. There are four types of aggregate gradation: well-graded, uniformly (or one-sized) graded, gap-graded, and open-graded.
Well-graded aggregates include a complete range of necessary size fractions that fit together to create a dense matrix with minimal voids, represented by a smooth, continuous gradation curve. This type of grading ensures good...
Well-graded aggregates include a complete range of necessary size fractions that fit together to create a dense matrix with minimal voids, represented by a smooth, continuous gradation curve. This type of grading ensures good...
428
Unsymmetric Loading of Thin-Walled Members
101
Thin-walled members with non-symmetrical cross-sections are vital to engineering structures, offering material efficiency and structural integrity. However, unsymmetrical loading on these members leads to complex stress distributions, resulting in simultaneous bending and twisting can cause deformation or structural failure. The interaction between bending and twisting requires detailed analysis to ensure structural resilience.
The concept of the shear center is crucial in countering the...
The concept of the shear center is crucial in countering the...
101
Distributed Loads: Problem Solving
628
Beams are structural elements commonly employed in engineering applications requiring different load-carrying capacities. The first step in analyzing a beam under a distributed load is to simplify the problem by dividing the load into smaller regions, which allows one to consider each region separately and calculate the magnitude of the equivalent resultant load acting on each portion of the beam. The magnitude of the equivalent resultant load for each region can be determined by calculating...
628


