在记忆神经网络中,层组合为故障耐受性的平均值
Osama Yousuf1,2,3, Brian D Hoskins2, Karthick Ramu2
1Department of Electrical and Computer Engineering, George Washington University, Washington, DC, USA.
Nature communications
|February 1, 2025
概括
层组合平均化增强了非理想的记忆神经网络,提高了计算中的推理性能. 这种容错方案提高了图像分类和持续学习任务的准确性.
科学领域:
- 计算机科学 计算机科学
- 材料科学 材料科学 材料科学
- 电气工程 电气工程
背景情况:
- 传统计算面临着内存瓶,限制了人工神经网络 (ANN) 的发展.
- 使用memristor设备进行内存计算显示出潜力,但受到硬件非理想性的困扰.
- 开发容错架构对于可靠的记忆神经网络至关重要.
研究的目的:
- 提出并验证一种以硬件为导向的容错方案,称为层组合平均.
- 提高非理想记忆神经网络的推断性能.
- 在模拟和硬件平台上展示该计划的有效性.
主要方法:
- 在记忆神经网络中实现层组合对故障容忍的平均值.
- 用编程的预训练解决方案进行图像分类任务的模拟.
- 在使用20,000个设备的原型设计平台,对持续学习问题进行硬件实验.
主要成果:
- 在图像分类和持续学习任务中观察到显著的性能增长.
- 对于有20%缺陷的图像分类,精度从40%提高到89.6%.
- 对于持续学习,准确性从55%提高到71%的水平,且性能开销最小.
结论:
- 层组合平均化有效地减轻了记忆神经网络中的硬件非理想性.
- 拟议的方案在类似的冗余级别上比以前的方法提供了显著的性能改进.
- 容错方法广泛适用于各种基于非挥发性设备的加速器.
相关概念视频
01:37Neural Regulation
Digestion begins with a cephalic phase that prepares the digestive system to receive food. When our brain processes visual or olfactory information about food, it triggers impulses in the cranial nerves innervating the salivary glands and stomach to prepare for food.
01:25Neural Circuits
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...