まず戻り,それから探検

Adrien Ecoffet^1,2, Joost Huizinga^3,4, Joel Lehman^5,6

¹Uber AI Labs, San Francisco, CA, USA. adrienecoffet@gmail.com.

Nature

|February 25, 2021

まとめ

Go-Exploreアルゴリズムは,状態を記憶し,探索する前にその状態に戻すことで補強学習を改善します. このアプローチは,これまで解明されていないゲームを解決し,AIの探索能力を向上させます.

科学分野:

人工知能
機械学習
ロボット

背景:

補強学習 (RL) は自律的な意思決定を目的としていますが,希少または欺瞞的な報酬信号に苦しんでいます.
効果的な環境調査はRLにとって極めて重要ですが,依然として大きな課題です.
既存の RL アルゴリズムは,以前訪問した状態を忘れ,新しい状態を探索する前に再訪問することに失敗します.

研究の目的:

強化学習の探索における脱線と脱線という課題に取り組むこと
Go-Exploreという新しいアルゴリズムを導入し,より効果的な環境探索を目的としています.
Go-Exploreが複雑なゲームやロボット作業に有効性を示すために

主な方法:

Go-Exploreというアルゴリズムファミリーを導入し,有望な状態を記憶し,探索前にそれらに戻ることを強調しました.
Go-Exploreを未解決のアタリゲームとハード探索のベンチマークに適用した.
"Go-Explore"を賞与の少ないピック&プレイスロボットでテストしました
探査の効率をさらに高めストキャスティシズムに対処するための統合された目標条件付きの政策

主要な成果:

Go-Exploreはこれまで解明されていないすべてのアタリゲームを解決しました.
ハードな探検ゲームで最先端のパフォーマンスを達成し,モンテズマの復讐とピットフォールに大幅に改善しました.
挑戦的な報酬の少ないロボティクスタスクの実践的な適用性を示した.
Go-Exploreの探査効率とストキャスティシティの強さを高めました

結論:

RLの探索には強力で一般的なアプローチです.
Go-Exploreは優れたパフォーマンスを提供しよりインテリジェントな学習エージェントへの重要な経路を示唆しています
RLの限界を克服するために,構造化された探査戦略の重要性を強調しています.

さらに関連する動画

08:28

Novel Object Exploration as a Potential Assay for Higher Order Repetitive Behaviors in Mice

Published on: August 20, 2016

8.7K

06:35

Examining Recall Memory in Infancy and Early Childhood Using the Elicited Imitation Paradigm

Published on: April 28, 2016

34.8K

関連する概念動画

01:12

First Pass Effect

7.8K

Presystemic elimination, or the first-pass effect, is the metabolism of drugs that reduces their effective concentration at the site of action. Apart from the first-pass effect, the systemic bioavailability of the drug is also reduced by other factors, including incomplete absorption or chemical degradation of drugs.
Depending on the route of administration, drugs can be metabolized in the liver, intestine, lungs, and vasculature. Orally administered drugs are first absorbed through the...

7.8K

01:12

Adjusting a Traverse

233

In the site survey of a four-sided traverse, internal angles are essential to ensure geometric accuracy. The survey revealed that the sum of the measured internal angles was 359 degrees and 48 minutes, which is 12 minutes less than the expected 360 degrees. This discrepancy signals an error likely arising from measurement inaccuracies during the fieldwork.To rectify this error, the adjustment process involved distributing the 12-minute shortfall equally across the four internal angles. By...

233

01:16

Crossover Experiments

4.3K

Crossover experiments, also called the repeated-measurements design, is a study design in which all experimental units are exposed to all treatments in different periods. Crossover experiments are generally used in psychology, the pharmaceutical industry, agriculture, and medicine.
Crossover designs are performed even with smaller sample sizes since the samples can act as their controls. These are better than simple randomized trials since patients are exposed to all the treatments.

4.3K

01:08

Reflex Activity

2.5K

A reflex activity is an automatic, involuntary response to specific stimuli. It is a part of our survival mechanism, designed to protect us from potential harm. For example, when a bright light suddenly shines into our eyes, we instinctively close them or look away. This is a simple reflex activity orchestrated by the nervous system without conscious thought or effort.
A reflex exam is a diagnostic procedure performed by a healthcare professional to evaluate the functionality of a patient's...

2.5K

01:29

Introspection

Introspection, long upheld as a reliable route to self-knowledge, involves examining one's thoughts, emotions, and mental processes. It underpins many psychological practices, from mindfulness meditation to psychotherapy and self-help strategies. However, empirical evidence challenges the accuracy of introspection as a means of understanding oneself.Limitations of Introspective InsightSeminal work by Nisbett and Wilson demonstrated that individuals are frequently unaware of the true causes...

02:31

Homologous Recombination

5.5K

関連する実験動画