刺激性培训++:超越剩余网络的性能限制
概括
剩余网络遭受"网络松",其中子网络表现不佳. 刺激性培训通过鼓励子网络更加努力工作,改善深度学习模型来提高绩效.
科学领域:
- 深度学习 (Deep Learning) 是一种深度学习.
- 计算机视觉 计算机视觉
- 人工智能的人工智能
背景情况:
- 在深度学习中,剩余网络至关重要.
- 它们可以被看作是浅层子网络的集体.
- 作为更大的网络的一部分,子网络可能表现不佳,这种现象被称为"网络分离".
研究的目的:
- 调查残余网络中的网络散情况.
- 提出一个新的培训计划,刺激性培训,以减轻网络乏.
- 提高剩余网络的性能,超出当前的限制.
主要方法:
- 引入了"网络松散"作为一个问题,子网络所付出的努力较少.
- 提出"刺激性培训",一个使用KL分歧损失用于子网络监督的方案.
- 开发了三个策略:KL-logit方向的损失,随机较小的输入和阶段间采样规则.
主要成果:
- 刺激性训练使ResNet50在ImageNet上提高到80.5%的Top1准确度,没有额外的数据或模型更改.
- 随着均增强,精度达到81.0%的Top1,超过了现有的基准.
- 在各种模型,数据集和任务中验证了有效性.
结论:
- 刺激性培训有效地解决了残余网络中的网络散问题.
- 提出的方法提供了显著的性能增长与最小的修改.
- 作为一个一般的,下一代的残余网络培训技术.
相关概念视频
01:35Long-term Potentiation
54.5K
Long-term potentiation, or LTP, is one of the ways by which synaptic plasticity—changes in the strength of chemical synapses—can occur in the brain. LTP is the process of synaptic strengthening that occurs over time between pre- and postsynaptic neuronal connections. The synaptic strengthening of LTP works in opposition to the synaptic weakening of long-term depression (LTD) and together are the main mechanisms that underlie learning and memory.
54.5K
01:27Exercise and Muscle Performance
1.3K
Exercise induces a range of adaptations in muscle tissue, depending on the type and duration of activity. Such physical training can be broadly categorized into two types: endurance exercises and resistance exercises.
Endurance exercises
Endurance exercises involve running, swimming, or cycling, which require repetitive movements with low force output. When a person engages in endurance exercise, a few noticeable changes occur in their skeletal muscles. For instance, the number of capillaries...
Endurance exercises
Endurance exercises involve running, swimming, or cycling, which require repetitive movements with low force output. When a person engages in endurance exercise, a few noticeable changes occur in their skeletal muscles. For instance, the number of capillaries...
1.3K