没有梯度的De Novo学习
Karl Friston1,2, Thomas Parr3, Conor Heins2
1Queen Square Institute of Neurology, University College London, London WC1E 6BT, UK.
Entropy (Basel, Switzerland)
|September 27, 2025
概括
这项研究引入了一种新的积极推理方法,用于新的目标导向行为的学习. 该方法发现了政策优化的生成模型,使代理商能够从头开始学习最佳行动.
科学领域:
- 计算神经科学是一种神经科学.
- 机器学习 机器学习
- 人工智能的人工智能
背景情况:
- 从头开始学习目标导向行为 (de novo学习) 是人工智能和神经科学的一个关键挑战.
- 当前的方法往往需要预先定义的模型或广泛的培训数据.
- 了解代理人如何在没有事先知识的情况下发现最佳政策对于开发更自主系统至关重要.
研究的目的:
- 应用主动推理来解决 de novo学习问题,以顺序优化政策.
- 开发一种程序,直接从观察中发现离散生成模型的结构和参数.
- 展示自由能源原理如何将学习重新定义为在生成模型动态中发现吸引集的发现.
主要方法:
- 积极推断框架应用于新的学习.
- 一个程序,增长和减少一个生成模型,以找到一个 pullback 吸引器在一般化状态.
- 使用自由能源原则重新构建学习问题.
- 与基于价值的公式进行比较,例如贝尔曼最佳性.
主要成果:
- 提出的方法成功地学习了一个具有吸引力的集合的生成模型,它代表了最少行动的路径.
- 这种吸引集引导代理人走向目标状态,同时避免昂贵的状态.
- 在模拟的街机游戏中展示了de novo结构学习和新兴机构.
结论:
- 积极推断为目标导向行为的新学习提供了一个有效的框架.
- 根据自由能源原则,可以将学习定义为在生成模型中发现有吸引力的集合.
- 这种方法有助于从观测数据中发现政策和代理行为.
相关概念视频
01:18Introduction to Learning
954
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
954
01:27Associative Learning
1.2K
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
1.2K
01:12Observational Learning
838
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
838
01:14Avoidance Learning and Learned Helplessness
2.5K
Avoidance learning and learned helplessness are critical concepts in understanding behavioral responses to negative stimuli.
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
2.5K
01:22Purposive Learning
444
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
444
01:21Cognitive Learning
1.0K
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
1.0K