Deep Reinforcement Learning With Explicit Context Representation.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Reinforcement Learning

Background:

• Reinforcement learning (RL) excels at complex problems but often struggles with contextual information.
• Human decision-making effectively utilizes context to avoid errors, a capability lacking in many RL agents.
• Current RL agents require extensive training to avoid obvious mistakes that humans identify through context.

Purpose of the Study:

• To propose a novel framework, Iota explicit context representation (IECR), for discrete environments.
• To enhance RL agents' ability to learn from contextual information.
• To improve learning efficiency and performance by incorporating explicit context.

Main Methods:

• Representing states using contextual key frames (CKFs).
• Extracting a function for state affordances from CKFs.
• Introducing two novel loss functions related to state affordances.
• Developing four new context-aware RL algorithms: IDQN, IDDQN, IDuDQN, IDDDQN.

Main Results:

• The IECR framework successfully extracts and learns from contextual information.
• All developed algorithms using IECR demonstrated significantly improved performance.
• Algorithms converged within approximately 40,000 neural network training steps, outperforming state-of-the-art equivalents.

Conclusions:

• The IECR framework provides an effective method for incorporating explicit context into RL.
• Contextual learning significantly accelerates convergence and improves performance in discrete environments.
• The proposed algorithms offer a substantial advancement for RL agents operating with contextual awareness.