转移学习与单数值分解的多通道卷积矩阵的转移学习
Tak Shing Au Yeung1, Ka Chun Cheung2,3, Michael K Ng4
1NVIDIA AI Technology Center, NVIDIA, Hong Kong 852, China iauyeung@nvidia.com.
Neural computation
|July 31, 2023
概括
这项研究引入了一种新的卷积-SVD层,用于卷积神经网络的转移学习. 该方法通过减少尺寸和微调单数值来提高预测准确度,以便更好地概括.
科学领域:
- 计算机科学 计算机科学
- 机器学习 机器学习
- 人工智能的人工智能
背景情况:
- 转移学习利用预训练的模型来提高对新任务的性能.
- 卷积神经网络 (CNN) 是图像分析的强大工具,但可能容易过度匹配.
- 分析卷积运算符是理解CNN行为的关键.
研究的目的:
- 提出一种新的卷积-SVD层,用于分析转移学习中的卷积运算符.
- 为了实现尺寸缩小,避免过度装配,同时保持微调的灵活性.
- 根据转移学习差距开发一个规范化模型.
主要方法:
- 单值值分解 (SVD) 在卷积运算符的福里埃域中计算.
- 从源域转移到目标域的奇点向量,并微调奇点值.
- 扩展卷积内核重建算法和设计泛化界限.
- 引入并利用转移学习差距作为调整器.
主要成果:
- 拟议的卷积-SVD层实现了尺寸缩小,防止过拟合.
- 一个有限的概括证明了训练和测试错误之间的一致性.
- 规范化模型有效地使用转移学习差距来限制测试错误.
- 数字实验表明,在分类任务中表现优越.
结论:
- 卷积-SVD层为CNN中转移学习提供了一种有效的方法.
- 基于转移学习差距的规范化显著提高了预测准确性.
- 该方法在缩小尺寸和模型灵活性之间提供了平衡.
相关概念视频
Multi-input and Multi-variable systems
129
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
129
Vector Algebra: Method of Components
14.0K
It is cumbersome to find the magnitudes of vectors using the parallelogram rule or using the graphical method to perform mathematical operations like addition, subtraction, and multiplication. There are two ways to circumvent this algebraic complexity. One way is to draw the vectors to scale, as in navigation, and read approximate vector lengths and angles (directions) from the graphs. The other way is to use the method of components.
In many applications, the magnitudes and directions of...
In many applications, the magnitudes and directions of...
14.0K
Deconvolution
189
Deconvolution, also known as inverse filtering, is the process of extracting the impulse response from known input and output signals. This technique is vital in scenarios where the system's characteristics are unknown, and they must be inferred from the observable signals.
Deconvolution involves several mathematical techniques to derive the impulse response. One common approach is polynomial division. In this method, the input and output sequences are treated as coefficients of...
Deconvolution involves several mathematical techniques to derive the impulse response. One common approach is polynomial division. In this method, the input and output sequences are treated as coefficients of...
189
Associative Learning
444
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
444
Convolution: Math, Graphics, and Discrete Signals
293
In any LTI (Linear Time-Invariant) system, the convolution of two signals is denoted using a convolution operator, assuming all initial conditions are zero. The convolution integral can be divided into two parts: the zero-input or natural response and the zero-state or forced response, with t0 indicating the initial time.
To simplify the convolution integral, it is assumed that both the input signal and impulse response are zero for negative time values. The graphical convolution process...
To simplify the convolution integral, it is assumed that both the input signal and impulse response are zero for negative time values. The graphical convolution process...
293
Convolution Properties I
182
Convolution computations can be simplified by utilizing their inherent properties.
The commutative property reveals that the input and the impulse response of an LTI (Linear Time-Invariant) system can be interchanged without affecting the output:
The commutative property reveals that the input and the impulse response of an LTI (Linear Time-Invariant) system can be interchanged without affecting the output:
182


