一目了然的策略:对优化早期退出深度神经网络的培训技术进行比较分析
Haseena Rahmath P1, Kuldeep Chaurasia1, Abhay Bansal1
1School of Computer Science Engineering and Technology, Bennett University, Plot Nos 8-11, TechZone II, Greater Noida, UP, India.
概括
这项研究比较了早期退出深度神经网络 (DNN) 的六种训练策略. 混合策略为自适应推理提供了准确性和计算效率的最佳平衡.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 早期退出深度神经网络 (DNN) 允许自适应推断,通过在中间层进行预测来降低计算成本.
- 早期退出DNN的表现受到培训策略的重大影响,这些策略尚未被系统地探索.
研究的目的:
- 系统地比较六个主要的早期退出DNN的培训策略.
- 评估这些策略对各种架构和数据集的准确性,收性,过拟合性和培训效率的影响.
主要方法:
- 对比联合,单独,分支,两阶段,蒸式和混合战略.
- 在CIFAR-10,CIFAR-100,ImageNet-100和CastX-ray14基准上使用了MobileNet,ResNet和VGG架构.
- 采用统计验证,包括ANOVA和Tukey的HSD测试.
主要成果:
- 基于联合和蒸的策略显示出强烈的泛化,但计算成本更高.
- 两个阶段和分支明智的策略在更深的退出处表现出过度装配;在早期退出时,单独的培训表现不佳.
- 混合策略在准确性和效率之间取得了卓越的平衡.
结论:
- 培训策略的选择极大地影响了早期退出DNN的表现.
- 混合策略为资源有限的应用提供了最佳的权衡.
- 结果为优化DNN培训提供了实际指导,并为未来的研究提供了信息.
相关概念视频
01:19Survival Tree
159
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...
Building a Survival Tree
Constructing a...
159
02:07Improving Translational Accuracy
11.9K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
11.9K