修剪和量子化算法与基于memristor的卷积神经网络中的应用
Mei Guo1, Yurui Sun1, Yongliang Zhu1
1College of Electrical Engineering and Automation, Shandong University of Science and Technology, Qingdao, 266590 China.
Cognitive neurodynamics
|February 26, 2024
概括
这项研究介绍了一种新的基于memristor的卷积神经网络,使用SBT-memristor和混合优化技术. 新架构显著降低了memristor数量,功耗和模型大小,以实现高效的AI应用.
科学领域:
- 神经形态工程的神经形态工程
- 人工智能的人工智能
- 材料科学 材料科学 材料科学
背景情况:
- 基于memristor的卷积神经网络 (CNN) 模仿大脑的效率,但面临着日益复杂的挑战.
- 更大的网络需要更多的memristor,导致更高的功耗和更大的模型尺寸.
- 现有的memristor CNNs难以适应先进应用的规模.
研究的目的:
- 提出一个基于SBT-memristor的CNN架构.
- 为memristor CNNs开发一种混合优化方法,将修剪和量子化结合起来.
- 为了降低memristor CNNs的尺寸和功耗,同时保持性能.
主要方法:
- 使用memristor的值属性构建了一个基于SBT的CNN.
- 设计了内存内存计算,激活和最大共享单元.
- 应用了一种混合优化技术,整合了网络修剪和重量量化.
主要成果:
- 拟议的架构显著减少了所需的memristors的数量.
- 实现了更低的功耗和压缩网络模型.
- 在MNIST数据集上表现出更快的识别速度和更低的功耗,精度损失最小.
结论:
- 基于SBT-memristor的CNN为节能和紧的AI系统提供了可行的解决方案.
- 混合优化有效地简化了memristor CNNs,并改善了重量表示.
- 这种方法为资源有限的环境中复杂的CNN应用提供了一条途径.
相关概念视频
01:01Neuroplasticity
359
Neuroplasticity reflects the brain's remarkable capacity to adapt and evolve, responding dynamically to learning, experiences, or injury by reorganizing its neural circuitry. This reorganization involves creating new neural connections and refining old ones through a series of biological processes that contribute to the brain's lifelong development and adaptability.
359
01:25Neural Circuits
1.2K
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
1.2K
01:22Upsampling
236
Managing signal sampling rates is essential in digital signal processing to maintain signal integrity. A decimated signal, characterized by a reduced frequency range due to its lower sampling rate, can be upsampled by inserting zeros between each sample. This upsampling process expands the original spectrum and introduces repeated spectral replicas at intervals dictated by the new Nyquist frequency. To refine this zero-inserted sequence, it is passed through a lowpass filter with a cutoff...
236
01:20Downsampling
157
When considering a sampled sequence with zero values between sampling instants, one can replace it by taking every N-th value of the sequence. At these integer multiples of N, the original and sampled sequences coincide. This process, known as decimation, involves extracting every N-th sample from a sequence, thereby creating a more efficient sequence.
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
157