PowerPlay: Training an Increasingly General Problem Solver by Continually Searching for the Simplest Still Unsolvable...

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Theory

Background:

• Traditional AI focuses on solving predefined problems.
• There is a need for AI systems that can autonomously discover and learn new problems.
• Unsupervised learning and creative problem generation are key to developing more general AI.

Purpose of the Study:

• To introduce and explore the PowerPlay algorithmic framework for automated problem discovery.
• To train increasingly general problem solvers from scratch using unsupervised learning.
• To investigate AI creativity inspired by playful learning behaviors.

Main Methods:

• PowerPlay searches for new tasks and solver modifications to create more powerful problem solvers.
• It prioritizes tasks that enhance efficiency (time/space) and can be validated quickly.
• The framework uses greedy search and time-optimal program search for efficient exploration.

Main Results:

• PowerPlay continuously breaks the generalization abilities of its current solver by seeking novelty.
• Newly invented tasks build upon previously learned skills, allowing for efficient reuse.
• The computational cost of validating new tasks does not necessarily increase with the task repertoire size.

Conclusions:

• PowerPlay offers a practical approach to implementing principles of AI creativity and self-improvement.
• The framework enables the development of AI systems capable of unsupervised, creative problem-solving.
• This approach contrasts with standard AI generalization, fostering continuous AI advancement.