在具有局部错误信号的SNN中,可学习的泄漏和开启的自我注意力
Cong Shi1,2, Li Wang1, Haoran Gao1
1School of Microelectronics and Communication Engineering, Chongqing University, Chongqing 400044, China.
Sensors (Basel, Switzerland)
|December 23, 2023
概括
这项研究通过优化神经元结构和训练策略来增强深度尖端神经网络 (SNN). 这种新的方法在较少的时间步骤中实现了高精度,提高了SNN在实际应用中的效率.
科学领域:
- 计算神经科学是一种神经科学.
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 尖端神经网络 (SNN) 模仿生物神经处理,但在深层架构中面临着关于信息聚焦和平衡的时空特征转换的挑战.
- 深度SNN中的有效信息处理和特征转换仍然是关键的研究领域.
研究的目的:
- 加强深度SNN的结构和战略,以提高信息处理和培训效率.
- 解决深度SNN中有效信息聚焦和平衡的时空特征转换的挑战.
主要方法:
- 通过引入可学习的泄漏系数 (LLC) 来优化泄漏的整合和发射 (LIF) 神经元模型.
- 在最初的时间步骤中集成了一种自我注意机制,以提高信息集中度.
- 开发了一种基于LLC的新型规范化方法和本地损失信号策略,以改善培训.
主要成果:
- 提出的方法在MNIST,FashionMNIST和CIFAR-10数据集上得到了验证.
- 该SNN模型在仅八个时间步骤内展示了卓越,高精度的性能.
- 实验结果证实了结构和战略增强的有效性.
结论:
- 这项研究为SNN结构和战略提供了新的见解,特别是关于可学习的参数和注意力机制.
- 提议的改进带来了高效和强大的SNN,适合实际应用.
- 这项工作为在各种机器学习任务中更有效的深度SNN铺平了道路.
相关概念视频
Neural Regulation
Digestion begins with a cephalic phase that prepares the digestive system to receive food. When our brain processes visual or olfactory information about food, it triggers impulses in the cranial nerves innervating the salivary glands and stomach to prepare for food.
Long-term Potentiation
Long-term potentiation, or LTP, is one of the ways by which synaptic plasticity—changes in the strength of chemical synapses—can occur in the brain. LTP is the process of synaptic strengthening that occurs over time between pre- and postsynaptic neuronal connections. The synaptic strengthening of LTP works in opposition to the synaptic weakening of long-term depression (LTD) and together are the main mechanisms that underlie learning and memory.
Random and Systematic Errors
Scientists always try their best to record measurements with the utmost accuracy and precision. However, sometimes errors do occur. These errors can be random or systematic. Random errors are observed due to the inconsistency or fluctuation in the measurement process, or variations in the quantity itself that is being measured. Such errors fluctuate from being greater than or less than the true value in repeated measurements. Consider a scientist measuring the length of an earthworm using a...
Types of Errors: Detection and Minimization
Error is the deviation of the obtained result from the true, expected value or the estimated central value. Errors are expressed in absolute or relative terms.
Absolute error in a measurement is the numerical difference from the true or central value. Relative error is the ratio between absolute error and the true or central value, expressed as a percentage.
Errors can be classified by source, magnitude, and sign. There are three types of errors: systematic, random, and gross.
Systematic or...
Absolute error in a measurement is the numerical difference from the true or central value. Relative error is the ratio between absolute error and the true or central value, expressed as a percentage.
Errors can be classified by source, magnitude, and sign. There are three types of errors: systematic, random, and gross.
Systematic or...
Detection of Gross Error: The Q Test
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
Random and Systematic Errors
Scientists always try their best to record measurements with the utmost accuracy and precision. However, sometimes errors do occur. These errors can be random or systematic. Random errors are observed due to the inconsistency or fluctuation in the measurement process, or variations in the quantity itself that is being measured. Such errors fluctuate from being greater than or less than the true value in repeated measurements. Consider a scientist measuring the length of an earthworm using a...


