赫比亚血统:关于日志-概率学习的统一观点
Jan Melchior1, Robin Schiewer2, Laurenz Wiskott3
1Ruhr University Bochum, 44801 Bochum, Germany jan.melchior@ini.rub.de.
Neural computation
|August 20, 2024
概括
赫比亚式下降是一种人工神经网络的新方法,通过忽视输出层中的激活函数导数来克服持续学习的局限性. 这种方法提高了性能,并防止了在深层和浅层网络中的灾难性遗忘.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 计算神经科学是一种神经科学.
背景情况:
- 人工神经网络输出层中的激活函数的导数可能会对性能产生负面影响,特别是在持续学习场景中.
- 在和激活区域的反向传播过程中消失的错误信号对训练神经网络构成了挑战.
研究的目的:
- 提出Hebbian下降作为一个理论框架和实际实施,以解决由输出层激活函数衍生物引起的局限性.
- 为了引入一个替代的损失函数,赫比亚式下降损失,用于绕过激活函数导数的梯度下降.
主要方法:
- 开发了Hebbian descent,这是输出层的替代权重更新规则,不考虑激活函数的衍生值.
- 实现了Hebbian下降损失,相当于概括的日志概率损失.
- 在浅层和深层神经网络中评估了Hebbian下降,特别是在持续学习任务中,有和没有中心化.
主要成果:
- 赫比亚式下降有效地避免了和激活区域中消失的错误信号.
- 在浅层网络的持续学习中,Hebbian下降超越了Hebbian学习和其他更新规则,表现相当于梯度下降.
- 与其他方法相比, Hebbian 降落证明了对灾难性干扰的优越预防.
- 对于深度神经网络,赫比亚式下降表现出与标准梯度下降相似或更好的性能.
结论:
- 赫比亚语下降提供了关于赫比亚语学习,梯度下降和通用线性模型的统一视角.
- 拟议的方法增强了持续学习能力,并为训练神经网络提供了强大的替代方案,特别是那些在输出层中具有和激活函数的神经网络.
- 赫比亚系降落方便设计有效的损失激活功能组合,以改善神经网络训练.
更多相关视频
08:05Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
Published on: June 30, 2020
7.5K
08:04Fully Automated Leg Tracking in Freely Moving Insects using Feature Learning Leg Segmentation and Tracking FLLIT
Published on: April 23, 2020
6.8K
相关概念视频
Residuals and Least-Squares Property
7.3K
The vertical distance between the actual value of y and the estimated value of y. In other words, it measures the vertical distance between the actual data point and the predicted point on the line
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
7.3K
Associative Learning
322
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
322
Law of Segregation
65.4K
When crossing pea plants, Mendel noticed that one of the parental traits would sometimes disappear in the first generation of offspring, called the F1 generation, and could reappear in the next generation (F2). He concluded that one of the traits must be dominant over the other, thereby causing masking of one trait in the F1 generation. When he crossed the F1 plants, he found that 75% of the offspring in the F2 generation had the dominant phenotype, while 25% had the recessive phenotype.
65.4K
Law of Independent Assortment
55.2K
While Mendel’s Law of Segregation states that the two alleles for one gene are separated into different gametes, a different question of how different genes are inherited remains. For example, is the gene for tall plants inherited with the gene for green peas? Mendel asked this question by experimenting with a dihybrid cross; a cross in which both parents are homozygous for two distinct traits resulting in an F1 generation that are heterozygous for both traits.
55.2K
Behavioral Genetics and Its Designs
346
Behavior genetics explores how genetic inheritance influences human behavior. It focuses on how genes, passed from parents to offspring, contribute to the development of behavioral traits and tendencies. This branch of genetics seeks to understand the complex interplay between inherited genetic factors and environmental influences in shaping our behaviors.
The primary methodologies used in behavior genetics include family studies, twin studies, and adoption studies, each providing unique...
The primary methodologies used in behavior genetics include family studies, twin studies, and adoption studies, each providing unique...
346
Heritability
195
Heritability is a statistical concept that measures the degree to which genetic differences among individuals contribute to trait variations within a population. It is a fundamental idea in genetics, often prone to misinterpretation. Heritability is expressed as a percentage, reflecting the proportion of variation in a specific trait across a population that can be linked to genetic differences. However, it's important to understand that heritability does not determine how "genetic"...
195
