ALBSNN:超低延迟自适应局部二进制尖端神经网络,具有准确性损失估计器
Yijian Pei1, Changqing Xu1,2, Zili Wu3
1Guangzhou Institute of Technology, Xidian University, Xi'an, China.
Frontiers in neuroscience
|September 29, 2023
概括
这项研究介绍了一个自适应局部二进制尖端神经网络 (ALBSNN),它平衡了量子化和精度,以实现高效的AI. 这种新的方法显著减少了内存的使用量,同时保持了高的识别能力,即使是最小的时间步骤.
科学领域:
- 人工智能的人工智能
- 计算神经科学是一种神经科学.
- 深度学习 (Deep Learning) 是一种深度学习.
背景情况:
- 尖端神经网络 (SNN) 提供能源效率和时空处理,但面临越来越深的记忆挑战.
- 量子化,特别是二进制化,对于减少SNN内存足迹至关重要,但往往会损害准确性.
研究的目的:
- 开发一个超低延迟的自适应局部二进制尖端神经网络 (ALBSNN),尽量减少内存使用,同时保持分类准确性.
- 解决量子化程度和SNN的性能之间的权衡问题.
主要方法:
- 引入了一个具有准确性损失估计器的自适应本地二元化策略,以动态选择二元化层.
- 集成的全球平均聚合 (GAP) 取代完全连接的层,加速网络培训.
- 拟议的二元重量优化 (BWO) 直接调整二元重量,减轻量化错误和训练瓶.
主要成果:
- 与全精度网络相比,实现了超过20%的存储空间减少.
- 保持了相当于完全精确网络的识别能力.
- 在静态数据集 (例如, Fashion-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST 时尚-MNIST) 和神经形态数据集上证明了高准确度.
结论:
- 拟议的ALBSNN有效地平衡了量化和识别能力,在存储和培训时间方面提供了显著的优势.
- 使用自适应二元化和优化技术,SNN可以在较少的时间步骤和减少内存的情况下实现高精度.
相关概念视频
Reducing Line Loss
168
In a three-phase circuit, line loss is an indicator of energy dissipated as heat due to the resistance of transmission lines. To address this, incorporating transformers into the system—a step-up transformer at the source and a step-down transformer at the load—is a strategic solution. Two three-phase transformers are introduced to improve this.
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
168
Improving Translational Accuracy
11.5K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
11.5K


