快速转移学习方法使用随机层结和特征精细化策略
IEEE transactions on cybernetics
|October 30, 2024
概括
本研究介绍了一种基于摩尔-罗斯反向 (MPI) 的诱导转移学习 (ITL) 在深 convolutional 神经网络 (DCNNs) 的快速再培训策略. 新方法显著加快了参数微调的收速度,使ITL变得更加实用.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 计算机视觉 计算机视觉
背景情况:
- 诱导转移学习 (ITL) 使用预训练的深卷积神经网络 (DCNNs).
- 完全连接 (FC) 层的基于摩尔-罗斯逆 (MPI) 的参数微调是最近的ITL方法.
- 目前基于MPI的ITL方法面临着由于高计算需求的挑战,这限制了实际应用.
研究的目的:
- 开发一种新的快速再培训策略,以提高基于MPI的ITL的适用性.
- 解决现有的基于MPI的ITL方法的计算瓶.
- 为了提高DCNN中参数微调的收速度.
主要方法:
- 在重新培训时期实施随机层结协议,以管理功能改进.
- 纳入基于MPI的方法,在批处理过程中改进FC层参数.
- 在ImageNet上评估了战略,预训练了基准DCNNs.
主要成果:
- 拟议的ITL策略与传统ITL方法相比,实现了竞争性表现.
- 证明了显著改善的收速度.
- 在ResNet-50上使用动量随随机梯度下降 (SGDM) 实现了接近1.5倍快于标准再训练的收.
结论:
- 新的快速再培训战略有效地提高了基于MPI的ITL的实用性.
- 该方法为微调DCNN提供了一个计算效率高的替代方案.
- 这种方法为加速深度学习模型适应提供了一个有希望的方向.
相关概念视频
02:07Improving Translational Accuracy
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
02:07Improving Translational Accuracy
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...