Compositional RL Agents That Follow Language Commands in Temporal Logic.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Reinforcement learning (RL) agents traditionally struggle with generalizing to new tasks or complex command structures.
• Executing commands specified in formal languages like linear temporal logic (LTL) presents a significant challenge for current RL models.

Purpose of the Study:

• To develop a novel reinforcement learning framework capable of understanding and executing commands specified in linear temporal logic (LTL).
• To enable zero-shot generalization of RL agents to unseen, more complex formulas and diverse problem domains.

Main Methods:

• Utilized compositional recurrent neural networks structured according to the parse of LTL formulas.
• Implemented a multi-task learning approach allowing agents to learn from diverse tasks and generalize without additional training.
• Demonstrated the approach in symbolic, Minecraft-like, and Fetch robotic environments with discrete and continuous state-action spaces.

Main Results:

• The proposed method successfully enabled RL agents to execute LTL commands across various domains.
• Achieved zero-shot generalization to significantly more complex formulas than those encountered during training.
• The compositional network structure proved effective for multi-task learning and generalization.

Conclusions:

• The compositional recurrent neural network approach offers a powerful method for RL agents to interpret and execute complex, structured commands.
• This framework facilitates zero-shot generalization in RL, significantly enhancing agent adaptability and learning efficiency.
• The presented compositional structures are domain-agnostic, paving the way for broader applications in compositional AI.