Open-Ended Learning: A Conceptual Framework Based on Representational Redescription.

Area of Science:

• Artificial Intelligence
• Robotics
• Machine Learning

Background:

• Reinforcement learning (RL) typically requires a known domain with defined states, actions, and rewards, often formalized as Markov Decision Processes (MDPs).
• Open-ended learning presents a challenge where agents must learn across an unbounded sequence of unknown tasks, precluding pre-defined MDPs.
• Agents need to autonomously discover relevant state and action representations to solve novel tasks effectively.

Purpose of the Study:

• To propose a conceptual framework for open-ended learning in agents with low-level perception and action capabilities.
• To address the challenge of learning appropriate state and action representations when they are not given.
• To enable agents to cast unknown tasks as MDPs for solving via RL.

Main Methods:

• The proposed framework assumes an agent with basic perception and action capabilities receiving external task rewards.
• The agent must discover state and action representations to formulate tasks as MDPs.
• An iterative approach based on successive Representational Redescription processes is suggested.

Main Results:

• The relevance of discovered state and action representations is critical for efficient learning.
• Open-ended learning is framed as the challenge of building adequate representations through redescription.
• Intrinsic motivations are highlighted as key to overcoming challenges in representation building.

Conclusions:

• The framework facilitates open-ended learning by enabling agents to discover task-relevant representations.
• Iterative Representational Redescription, guided by intrinsic motivations, is a viable approach for agents learning in unknown domains.
• This work advances the understanding of how agents can learn and adapt in complex, unbounded environments.