Emergent Solutions to High-Dimensional Multitask Reinforcement Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Neuroscience

Background:

• Reinforcement learning (RL) algorithms struggle with dynamic, high-dimensional, and partially observable environments.
• Deep learning frameworks address high-dimensional data but are computationally demanding and complex.
• Existing RL methods often rely on pre-engineered features, limiting adaptability.

Purpose of the Study:

• To introduce a novel framework based on genetic programming for adaptive policy complexification in RL.
• To compare the proposed genetic programming approach against deep reinforcement learning and traditional RL methods.
• To evaluate the scalability and efficiency of the new framework in complex environments.

Main Methods:

• Developed a genetic programming framework that adaptively increases policy complexity through environmental interaction.
• Benchmarked the framework against deep reinforcement learning algorithms on Atari video games.
• Compared performance with traditional RL frameworks utilizing engineered features.

Main Results:

• The genetic programming approach achieved performance comparable to deep learning methods.
• The proposed framework demonstrated a minimum of three orders of magnitude reduction in model complexity.
• The system achieved real-time operation without specialized hardware and evolved solutions for multiple games concurrently.

Conclusions:

• Genetic programming provides a computationally efficient and simpler alternative to deep learning for reinforcement learning.
• The framework's ability to adapt and scale to multiple tasks simultaneously offers significant advantages.
• This approach enables real-time performance and reduces the need for extensive computational resources.