相关实验视频

Updated: Jun 18, 2025

Recording Single Neurons' Action Potentials from Freely Moving Pigeons Across Three Stages of Learning

Recording Single Neurons' Action Potentials from Freely Moving Pigeons Across Three Stages of Learning

Published on: June 2, 2014

对于强化学习的顺序动作诱导的不变表示

Dayang Liang¹, Qihang Chen¹, Yunlong Liu¹

¹Department of Automation, Xiamen University, Xiamen 361005, China.

Neural networks : the official journal of the International Neural Network Society

|August 3, 2024

概括

本研究介绍了顺序动作诱导的不变表示 (SAR),这是一种用于视觉强化学习的新方法. 通过利用动作序列,SAR有效地从有分心的观测中提取与任务相关的信息.

科学领域:

人工智能的人工智能
机器学习机器学习
机器人技术机器人技术机器人技术

背景情况:

从高维,视觉分散注意力的观测中学习与任务相关的状态表示是视觉增强学习的一个关键挑战.
现有的无监督表示学习方法 (双模拟,对比,预测,重建) 在处理分心和稀疏奖励方面面临局限性.

研究的目的:

在视觉分散注意力的环境中开发一种强大的方法来提取与任务相关的状态表示.
通过有效地分离与任务相关的和无关的信息来提高强化学习代理的性能.

主要方法:

提出序列动作诱导的不变表示 (SAR),将动作序列纳入表示学习.
模拟动作序列概率分布的特征函数,以优化状态编码器.
在使用顺序行动的观察中,将受控 (与任务相关) 和不受控制 (与任务无关) 的信息分离出来.

主要成果:

在分散注意力的DeepMind控制套件上实现了最先进的性能,超过了强大的基线.
在现实世界的自动驾驶场景 (CARLA) 中表现出有效性,具有自然分心.
通过泛化衰变和t-SNE可视化进行的分析证实了该方法能够忽略不相关信息.

结论:

SAR有效地从杂的观测中提取与任务相关的表示,即使有显著的视觉分心.
该方法显示出强大的概括能力和适用于现实世界的问题,如自动驾驶.
利用动作序列是改善对象学习在具有挑战性的强化学习领域的一个有希望的方向.

关键词:

行动顺序行动顺序代表性的学习学习.视觉分散注意力视觉分散注意力视觉增强学习的学习方法

更多相关视频

The "Motor" in Implicit Motor Sequence Learning: A Foot-stepping Serial Reaction Time Task

The "Motor" in Implicit Motor Sequence Learning: A Foot-stepping Serial Reaction Time Task

Published on: May 3, 2018

Quantifying Learning in Young Infants: Tracking Leg Actions During a Discovery-learning Task

Quantifying Learning in Young Infants: Tracking Leg Actions During a Discovery-learning Task

Published on: June 1, 2015

相关概念视频

State Space Representation

State Space Representation

The frequency-domain technique, commonly used in analyzing and designing feedback control systems, is effective for linear, time-invariant systems. However, it falls short when dealing with nonlinear, time-varying, and multiple-input multiple-output systems. The time-domain or state-space approach addresses these limitations by utilizing state variables to construct simultaneous, first-order differential equations, known as state equations, for an nth-order system.
Consider an RLC circuit, a...

Reinforcement Schedules

Reinforcement Schedules

Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...

Transfer Function to State Space

Transfer Function to State Space

State-space representation is a powerful tool for simulating physical systems on digital computers, necessitating the conversion of the transfer function into state-space form. Consider an nth-order linear differential equation with constant coefficients, like those encountered in an RLC circuit. The state variables are selected as the output and its n−1 derivatives. Differentiating these variables and substituting them back into the original equation produces the state equations.
In an...

Linear time-invariant Systems

Linear time-invariant Systems

A system is linear if it displays the characteristics of homogeneity and additivity, together termed the superposition property. This principle is fundamental in all linear systems. Linear time-invariant (LTI) systems include systems with linear elements and constant parameters.
The input-output behavior of an LTI system can be fully defined by its response to an impulsive excitation at its input. Once this impulse response is known, the system's reaction to any other input can be...

Fixed Action Patterns

Fixed Action Patterns

A fixed action pattern (FAP) is a specific, hard-wired sequence of behaviors that occurs in response to an external stimulus, called a sign stimulus. The behavior is “fixed” because it is essentially unchangeable—proceeding similarly across individuals of a species every time it occurs.

Instinctive Drift

Instinctive Drift

Instinctive drift refers to the tendency of animals to revert to their innate behaviors despite repeated reinforcement. Breland and Breland demonstrated this concept in an experiment with a raccoon. The raccoon was trained to pick up two coins and place them in a container in exchange for food. Initially, the raccoon learned to associate the coins with food, making them a conditioned stimulus or a substitute for food. However, over time, the raccoon became less willing to put the coins into the...