Decomposing user-defined tasks in a reinforcement learning setup using TextWorld.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Complex tasks pose significant challenges for training autonomous agents.
• Sparse reward functions in simulated environments often hinder effective learning.
• Hierarchical Reinforcement Learning (HRL) offers a potential solution for task decomposition.

Purpose of the Study:

• To propose and evaluate a novel hierarchical reinforcement learning (HRL) method for autonomous agent training.
• To demonstrate the benefits of task decomposition in improving agent learning efficiency.
• To leverage high-level abstractions for enhanced reward function design.

Main Methods:

• Implementation of a hierarchical reinforcement learning (HRL) framework using TextWorld and MiniGrid Python environments.
• Utilizing MiniGrid for 2D environment simulation and TextWorld for high-level task abstraction.
• Designing a dense reward function for the lower-level environment based on task abstraction.
• Employing formal methods to verify the solution-finding capabilities of the proposed algorithm.

Main Results:

• The proposed HRL method successfully decomposes complex tasks into manageable sub-tasks.
• Training on the TextWorld abstraction disentangles manipulation and navigation, simplifying the learning process.
• The use of a dense reward function significantly improves agent training performance compared to sparse rewards.
• Formal methods confirmed that the algorithm is capable of deriving solutions.

Conclusions:

• Hierarchical reinforcement learning (HRL) with task abstraction provides an effective strategy for training autonomous agents.
• The integration of TextWorld and MiniGrid facilitates a practical and efficient implementation of HRL.
• Disentangling actions and employing dense rewards are key factors in enhancing agent performance in simulated environments.