深層補強学習による人間レベルのコントロール
Volodymyr Mnih1, Koray Kavukcuoglu1, David Silver1
1Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
Nature
|February 27, 2015
まとめ
この研究は,エンドツーエンドの強化学習を使用して高次元の感覚入力から学習する人工的エージェントである深層のQネットワークを導入します. このエージェントは,Atariのゲームで人間レベルのパフォーマンスを達成し,未処理のピクセルデータから効果的な汎用化を実証しました.
科学分野:
- 人工知能 (AI) とは,人工知能 (AI) のことです.
- 機械学習 (Machine Learning) とは,機械学習 (Machine Learning) について学ぶことです.
- 計算神経科学とは
背景:
- 強化学習 (RL) は,心理学的および神経科学的な原則に基づいたエージェント制御を最適化します.
- 現実世界のRLは,一般化のために高次元の感覚インプットから効率的な表現を導き出すためにエージェントを必要とします.
- 既存のRLエージェントは,手作りした特徴や低次元,完全に観察された状態に限定されています.
研究 の 目的:
- 高次元の感覚インプットからエンドツーエンドの強化学習を行うことができる新しい人工エージェントを開発する.
- 複雑な現実世界のシナリオで以前のRLエージェントの限界を克服するために.
- 原始的な感覚データと人工エージェントにおける効果的な意思決定の間のギャップを埋めるために.
主な方法:
- ディープニューラルネットワークトレーニングの進歩を利用して,ディープQネットワークエージェントを作成しました.
- エンドツーエンドの強化学習を採用し,インプットとして原始ピクセルとゲームスコアのみを処理します.
- 49のクラシックアタリ2600ゲームの多様なセットでエージェントをテストしました.
主要な成果:
- ディープQネットワークエージェントは,アタリ2600ゲームにおけるこれまでのすべてのアルゴリズムを上回りました.
- テストされたゲーム全体で,プロのヒューマンゲームテスターに匹敵するパフォーマンスを達成しました.
- 高次元のビジュアル入力から直接学習と一般化を成功させることが実証されています.
結論:
- ディープQネットワークは,人工知能の重要な進歩であり,未処理の感覚データから学習することを可能にします.
- このアプローチは,高次元の入力とアクションの間の溝を橋渡しし,汎用的なエージェントを作成します.
- 様々な挑戦的なタスクでエージェントの成功は,深層補強学習の潜在力を強調しています.
関連する概念動画
01:23Reinforcement
1.2K
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
1.2K
01:06Law of Effect
6.0K
B.F. Skinner, a prominent figure in behavioral psychology, introduced operant conditioning by emphasizing the role of consequences in shaping behavior. This theory builds upon the law of effect proposed by Edward Thorndike, which posits that behaviors followed by satisfying outcomes are likely to be repeated. In contrast, those followed by unsatisfying outcomes are less likely to recur.
Edward Thorndike's foundational work involved studying learning in animals, particularly using puzzle...
Edward Thorndike's foundational work involved studying learning in animals, particularly using puzzle...
6.0K
01:12Observational Learning
1.4K
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
1.4K
01:14Avoidance Learning and Learned Helplessness
3.9K
Avoidance learning and learned helplessness are critical concepts in understanding behavioral responses to negative stimuli.
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
3.9K
01:24Reinforcement Schedules
721
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
721
01:18Hierarchy of Motor Control
6.9K
The hierarchy of motor control refers to the different levels of organization and processing involved in controlling movement in the body. These levels range from higher cortical areas involved in planning and decision-making to lower spinal cord reflexes that respond automatically to external stimuli.
6.9K