解构深度主动推理:一个相反的信息收集器
Théophile Champion1, Marek Grześ2, Lisa Bonheme3
1University of Birmingham, School of Computer Science Birmingham B15 2TT, U.K. txc314@student.bham.ac.uk.
Neural computation
|August 14, 2024
概括
使用深度学习开发了深度主动推理代理. 最大限度地奖励,而不是最大限度地减少预期的自由能量,使代理商能够通过鼓励探索来解决复杂的任务.
科学领域:
- 计算神经科学是一种计算神经科学.
- 机器学习是机器学习.
- 机器人技术 机器人技术 机器人技术
背景情况:
- 积极推断为感知,学习和决策提供了一个统一的理论.
- 深度学习和蒙特卡洛树搜索的进步旨在增强复杂任务的积极推断能力.
研究的目的:
- 开发和评估用于解决复杂任务的深度主动推理代理.
- 调查不同目标函数 (最小化预期的自由能量与最大化奖励) 对代理性能的影响.
- 在深度主动推理中分析认识价值的作用和制定.
主要方法:
- 实现一个变量自编码器 (VAE).
- 开发深度隐藏的马尔科夫模型 (HMMs),包括一个深度临界隐藏的马尔科夫模型 (CHMM).
- 实验CHMM版本,尽量减少预期的自由能量 (CHMM[EFE]) 和最大化奖励 (CHMM[奖励]),使用各种行动选择策略.
主要成果:
- 最大化奖励的CHMM代理成功解决了dSprites环境,与最小化预期自由能量的CHMM不同.
- 该CHMM[EFE]代理汇聚到一个单一的行动,阻碍了勘探和任务完成.
- 通过利用所有动作,CHMM[奖励]代理证明了有效的探索.
- 观察到恶化的认识价值表述,可能会减少信息获取.
结论:
- 在深度主动推理中,最大化奖励对于使探索和解决复杂任务至关重要.
- 深度主动推理中认识价值的制定需要进一步调查,以确保有效的信息获取和避免退化的行为.
- 该研究强调了在某些依赖勘探的任务中,尽量减少预期的自由能量的局限性.
更多相关视频
05:48The Adventures of Fundi Intervention Based on the Cognitive and Emotional Processing in Attention Deficit Hyperactive Disorder Patients
Published on: June 12, 2020
5.7K
05:41A Step-by-Step Implementation of DeepBehavior, Deep Learning Toolbox for Automated Behavior Analysis
Published on: February 6, 2020
9.4K
相关概念视频
01:16Deductive Reasoning
55.1K
Deductive reasoning, or deduction, is the type of logic used in hypothesis-based science. In deductive reasoning, the pattern of thinking moves in the opposite direction as compared to inductive reasoning, which means that it uses a general principle or law to predict specific results. From those general principles, a scientist can deduce and predict the specific results that would be valid as long as the general principles are valid.
For example, a researcher can deduce specific predictions...
For example, a researcher can deduce specific predictions...
55.1K
00:59Inductive Reasoning
60.2K
Inductive reasoning is a form of logical thinking that uses related observations to arrive at a general conclusion. It is uncertain and operates in degrees to which the conclusions are credible. As such, inductive arguments can be weak or strong, rather than valid or invalid, and conclusions can be used to formulate testable, falsifiable hypotheses.
Inductive reasoning is common in descriptive science. A life scientist makes observations and records them. This data can be qualitative or...
Inductive reasoning is common in descriptive science. A life scientist makes observations and records them. This data can be qualitative or...
60.2K
01:30Reasoning
72
Reasoning is the action of thinking about something in a logical, sensible way. It is integral to problem-solving, decision-making, and critical thinking. Reasoning can be inductive or deductive. Reasoning involves transforming information into conclusions, which is essential for problem-solving, decision-making, and critical thinking.
Inductive reasoning involves deriving generalizations from specific observations. This type of reasoning helps form beliefs about the world. For example,...
Inductive reasoning involves deriving generalizations from specific observations. This type of reasoning helps form beliefs about the world. For example,...
72
01:17Cognitivism
1.4K
Cognitive psychology emerged as a significant field in the mid-20th century. It focused on understanding humans' internal mental processes. This approach emphasizes how people perceive, remember, think, and solve problems—elements critical to human cognition.
Previously dominated by behaviorism, which prioritized observable behaviors and largely ignored mental processes, psychology transformed in the 1950s. Cognitive psychologists argue that understanding how we think and process...
Previously dominated by behaviorism, which prioritized observable behaviors and largely ignored mental processes, psychology transformed in the 1950s. Cognitive psychologists argue that understanding how we think and process...
1.4K
01:27Associative Learning
322
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
322
01:12Observational Learning
155
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
155