深度ART:深度无梯度本地学习与自适应共振
Sasha Petrenko1, Leonardo Enzo Brito da Silva2, Donald C Wunsch1
1Kummer Institute Center for Artificial Intelligence and Autonomous Systems, Missouri University of Science and Technology, Rolla, 65409, MO, USA.
概括
DeepART是一种新的无梯度方法,用于使用自适应共振理论 (ART) 训练深度神经网络. 这种方法通过高维数据提高了终身学习的性能和可扩展性.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 计算神经科学是一种神经科学.
背景情况:
- 深度神经网络在特征表示方面表现出色,但在终身学习方面扎.
- 适应共振理论 (ART) 算法提供了强大的终身学习,但可能是计算密集的.
- 将深度学习与ART集成,可以利用这两种模式的优势.
研究的目的:
- 介绍DeepART,一个无梯度的,一拍式增量学习技术,用于深度Hebbian神经网络.
- 为深度网络层调整FuzzyART模块,通过FuzzyARTMAP头部实现监督学习.
- 评估DeepART在使用高维数据集的终身学习场景中的有效性.
主要方法:
- 解释深度神经网络层作为修改后的FuzzyART模块,具有特定的输入编码和重量更新规则.
- 在完全连接层和卷积层中导出局部重量更新规则.
- 使用FuzzyARTMAP头进行特征类别标签映射,以促进监督学习.
主要成果:
- 与现有的基于ART的方法相比,DeepART实现了性能提升.
- 该技术有效地减少了类别的扩散,提高了可扩展性.
- 在终身学习环境中证明了对高维数据集的改进处理.
结论:
- 深度ART成功地将深度网络的特征学习与ART的终身学习能力相结合.
- 无梯度,一次性增量方法为复杂的学习任务提供了可扩展的解决方案.
- 这种方法为推进人工智能终身学习提供了一个有希望的方向.
相关概念视频
01:26Linear Approximation in Frequency Domain
139
Linear systems are characterized by two main properties: superposition and homogeneity. Superposition allows the response to multiple inputs to be the sum of the responses to each individual input. Homogeneity ensures that scaling an input by a scalar results in the response being scaled by the same scalar.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
139
01:12Differential Leveling
338
Differential leveling is a precise method in surveying used to determine the elevation difference between two points. Its primary goal is to establish accurate vertical measurements to create level surfaces or grade lines critical for designing and constructing infrastructures such as roads, bridges, and buildings.The procedure for differential leveling begins with setting up and leveling the instrument at a point where the benchmark can be seen. The level rod is held on the benchmark (BM), and...
338
01:27Associative Learning
605
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
605
01:11Residuals and Least-Squares Property
7.9K
The vertical distance between the actual value of y and the estimated value of y. In other words, it measures the vertical distance between the actual data point and the predicted point on the line
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
7.9K
01:18Introduction to Learning
551
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
551
01:18Neural Control of Respiration
3.0K
The neural regulation of respiration is a meticulously coordinated process primarily controlled by the respiratory centers located within the brainstem. These centers, composed of specialized neurons, transmit nerve impulses that control the contraction and relaxation of our respiratory muscles.
Respiratory Centers in the Brainstem
Two primary areas comprise the respiratory center: the medullary respiratory center in the medulla oblongata and the pontine respiratory group in the pons. The...
Respiratory Centers in the Brainstem
Two primary areas comprise the respiratory center: the medullary respiratory center in the medulla oblongata and the pontine respiratory group in the pons. The...
3.0K