収束神経ネットワークの圧縮のための減少したストレージ直接テンサーリング分解
1Faculty of Electronics, Photonics, and Microsystems, Wroclaw University of Science and Technology, Wybrzeze Wyspianskiego 27, Wroclaw, 50-370, Poland.
まとめ
この研究は,縮小されたストレージの直接テンソールリング分解 (RSDTR) を使用したコンボリューションニューラルネットワーク (CNNs) の圧縮のための新しい低ランク方法を導入しています. RSDTRは,高い画像分類精度を維持しながら,モデルのサイズと計算を大幅に削減します.
科学分野:
- コンピュータ・ビジョン
- 機械学習
- ディープラーニングの最適化
背景:
- コンボリューションニューラルネットワーク (CNN) は,画像分類のようなコンピュータビジョンのタスクに不可欠です.
- モデルの圧縮は,ストレージとコンピューティングの観点からCNNの効率を高めるために不可欠です.
- 低ランク近似方法は,大きなカーネルを分解することによって,CNN圧縮のための有望な道を提供します.
研究 の 目的:
- CNNの圧縮に新しい低ランク方法を提案します.
- 効率的なカーネルの近似のために,減少したストレージの直接テンサーリング分解 (RSDTR) を活用する.
- 高圧縮率を達成し,精度を維持するRSDTRの有効性を評価する.
主な方法:
- RSDTRに基づいた新しい低ランクのCNN圧縮技術を開発しました.
- パラメータとFLOPSの複雑さを減らすためにRSDTRを実装します.
- 性能を評価するためにCIFAR-10とImageNetのデータセットで実験を行った.
主要な成果:
- 提案されたRSDTR方法は,重要なパラメータとFLOPS圧縮率を達成しました.
- RSDTRは,既存の方法と比較して,より優れた円形モードパルムテーションの柔軟性を示しました.
- RSDTRを使用した圧縮ネットワークは,競争力のある分類精度を維持しました.
結論:
- RSDTRはCNNを圧縮するための効果的な方法です.
- このアプローチは,圧縮効率と分類性能の間の好ましいトレードオフを提供します.
- RSDTRは他の最先端のCNN圧縮技術よりも優れています
関連する概念動画
Convolution: Math, Graphics, and Discrete Signals
393
In any LTI (Linear Time-Invariant) system, the convolution of two signals is denoted using a convolution operator, assuming all initial conditions are zero. The convolution integral can be divided into two parts: the zero-input or natural response and the zero-state or forced response, with t0 indicating the initial time.
To simplify the convolution integral, it is assumed that both the input signal and impulse response are zero for negative time values. The graphical convolution process...
To simplify the convolution integral, it is assumed that both the input signal and impulse response are zero for negative time values. The graphical convolution process...
393
Convolution Properties I
232
Convolution computations can be simplified by utilizing their inherent properties.
The commutative property reveals that the input and the impulse response of an LTI (Linear Time-Invariant) system can be interchanged without affecting the output:
The commutative property reveals that the input and the impulse response of an LTI (Linear Time-Invariant) system can be interchanged without affecting the output:
232
Convolution Properties II
278
The important convolution properties include width, area, differentiation, and integration properties.
The width property indicates that if the durations of input signals are T1 and T2, then the width of the output response equals the sum of both durations, irrespective of the shapes of the two functions. For instance, convolving two rectangular pulses with durations of 2 seconds and 1 second results in a function with a width of 3 seconds.
The area property asserts that the area under the...
The width property indicates that if the durations of input signals are T1 and T2, then the width of the output response equals the sum of both durations, irrespective of the shapes of the two functions. For instance, convolving two rectangular pulses with durations of 2 seconds and 1 second results in a function with a width of 3 seconds.
The area property asserts that the area under the...
278
Reducing Line Loss
193
In a three-phase circuit, line loss is an indicator of energy dissipated as heat due to the resistance of transmission lines. To address this, incorporating transformers into the system—a step-up transformer at the source and a step-down transformer at the load—is a strategic solution. Two three-phase transformers are introduced to improve this.
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
193
Downsampling
250
When considering a sampled sequence with zero values between sampling instants, one can replace it by taking every N-th value of the sequence. At these integer multiples of N, the original and sampled sequences coincide. This process, known as decimation, involves extracting every N-th sample from a sequence, thereby creating a more efficient sequence.
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
250
Vector Operations
1.5K
Vectors are physical quantities that have both magnitude and direction. The vector operations include addition, subtraction, and scalar multiplication.
A vector multiplied by a scalar value is called scalar multiplication. The result obtained is a new vector with a different magnitude. If the scalar is positive, the direction of the vector remains the same, but if it is negative, the direction of the vector is reversed. For example, the product of the mass and velocity yields the momentum.
A vector multiplied by a scalar value is called scalar multiplication. The result obtained is a new vector with a different magnitude. If the scalar is positive, the direction of the vector remains the same, but if it is negative, the direction of the vector is reversed. For example, the product of the mass and velocity yields the momentum.
1.5K

