On the convergence of projective-simulation-based reinforcement learning in Markov decision processes

Area of Science:

• Quantum Machine Learning
• Artificial Intelligence
• Reinforcement Learning Theory

Background:

• Growing interest in quantum computing for machine learning tasks.
• Projective simulation as a novel framework for quantum-enhanced reinforcement learning.
• Limited formal theoretical analysis of projective simulation's performance.

Purpose of the Study:

• To provide a detailed formal discussion of projective simulation's properties.
• To analyze the convergence of projective simulation in reinforcement learning scenarios.
• To establish theoretical guarantees for a physically inspired reinforcement learning approach.

Main Methods:

• Formal theoretical analysis of the projective simulation model.
• Mathematical proof of convergence for a specific version of the model.
• Evaluation within the context of Markov decision processes.

Main Results:

• Convergence of the projective simulation model to optimal behavior is proven.
• Demonstration of guaranteed convergence for this reinforcement learning approach.
• Formal validation of projective simulation's theoretical performance.

Conclusions:

• Projective simulation, a quantum-inspired method, offers theoretically guaranteed convergence.
• Physically inspired approaches can yield provably effective reinforcement learning algorithms.
• This work establishes a foundation for further research into quantum-enhanced reinforcement learning.