在神经网络中,贝叶斯的持续学习和遗忘
Djohan Bonnet1, Kellian Cottart1, Tifenn Hirtzlin2
1Centre de Nanosciences et de Nanotechnologies, Université Paris-Saclay, CNRS, Palaiseau, France.
Nature communications
|October 31, 2025
概括
我们介绍了来自突触不确定性 (MESU) 的元可塑性,这是一个新的贝叶斯学习规则. MESU使人工神经网络能够在不遗忘的情况下不断学习,模仿生物突触,实现强大的,永恒的学习.
科学领域:
- 人工智能的人工智能
- 计算神经科学是一种神经科学.
- 机器学习 机器学习
背景情况:
- 人工神经网络 (ANN) 与生物突触不同,与灾难性的遗忘和记忆作斗争.
- 现有的方法缺乏一个原则性的方法来平衡记忆保留和灵活性.
研究的目的:
- 介绍从突触不确定性 (MESU) 引入元可塑性,这是ANN中持续学习的贝叶斯更新规则.
- 使ANN能够结合学习和忘记,而没有明确的任务界限,灵感来自生物突触.
主要方法:
- 开发了MESU,这是一个贝叶斯更新规则,通过不确定性扩展参数学习.
- 集成的认识系统不确定性估计用于分布外检测.
- 使用重量采样用于预测统计计算.
主要成果:
- 在图像分类基准中,MESU减轻了遗忘,同时保持了可塑性.
- 在连续的Permuted-MNIST任务上表现优于既定的突触整合方法.
- 在任务增量CIFAR-100.0中表现出比传统技术更优异的性能.
结论:
- MESU提供了一种以生物为灵感的方法,以在ANN中实现强大的,永恒的学习.
- 连接了元可塑性,贝叶斯推理和基于赫西安的规范化.
- 为ANN提供了一条途径,以实现类似于生物系统的持续学习.
相关概念视频
Forgetting
357
Forgetting is an intrinsic aspect of human memory, characterized by the gradual loss or inaccessibility of information over time. Hermann Ebbinghaus, a pioneering psychologist, extensively studied this phenomenon and formulated the forgetting curve. This curve illustrates that memory loss occurs rapidly immediately after learning and then decelerates over time. Several mechanisms contribute to forgetting, including encoding failure, storage decay, retrieval failure, and interference.
Encoding...
Encoding...
357
Observational Learning
807
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
807
Interference and Decay
407
Forgetting is a complex cognitive phenomenon influenced by several factors, among which interference and decay are particularly prominent. These processes explain why individuals often struggle to retrieve specific information from memory, leading to lapses in recall that can be observed in everyday situations.
Interference occurs when competing memories hinder the retrieval of particular information. It can be classified into two types: proactive and retroactive interference. Proactive...
Interference occurs when competing memories hinder the retrieval of particular information. It can be classified into two types: proactive and retroactive interference. Proactive...
407
Associative Learning
1.2K
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
1.2K
Neural Circuits
2.6K
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
2.6K
Long-term Potentiation
58.2K
Long-term potentiation, or LTP, is one of the ways by which synaptic plasticity—changes in the strength of chemical synapses—can occur in the brain. LTP is the process of synaptic strengthening that occurs over time between pre- and postsynaptic neuronal connections. The synaptic strengthening of LTP works in opposition to the synaptic weakening of long-term depression (LTD) and together are the main mechanisms that underlie learning and memory.
58.2K


