Fast and Epsilon-Optimal Discretized Pursuit Learning Automata

Area of Science:

• Artificial Intelligence
• Machine Learning
• Reinforcement Learning

Background:

• Learning automata (LA) are key tools in reinforcement learning.
• Discretized pursuit LA is a popular but slow method for large action sets.
• Current LA methods face slow learning due to extensive updates in action selection and state probability updates.

Purpose of the Study:

• To introduce a novel fast discretized pursuit LA.
• To ensure assured ε-optimality for the proposed LA.
• To address the computational inefficiency of traditional LA in large-scale action scenarios.

Main Methods:

• Developed a new discretized pursuit LA algorithm.
• Designed the algorithm to make computational complexity independent of the number of actions.
• Focused on optimizing action selection and state probability update phases.

Main Results:

• The proposed fast LA achieves computational complexity independent of the action space size.
• Demonstrated faster convergence speeds compared to classical LA in stationary environments.
• Ensured assured ε-optimality for the learning process.

Conclusions:

• The fast discretized pursuit LA offers significant improvements for large-scale reinforcement learning.
• This advancement promotes LA applications in areas needing efficient, optimal, and fast learning.
• The method provides a computationally efficient alternative for complex reinforcement learning tasks.