Reinforcement Learning With Sparse-Executing Action via Sparsity Regularization.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Reinforcement learning (RL) excels in decision-making but struggles with limited action budgets.
• Classical RL assumes unlimited action execution, failing in scenarios with sparse actions.

Purpose of the Study:

• To address the challenge of limited action execution in reinforcement learning.
• To introduce a novel algorithm for optimizing policies under action constraints.

Main Methods:

• Formalized the problem as a sparse action Markov decision process (SA-MDP).
• Proposed Action Sparsity REgularization (ASRE), a policy optimization algorithm.
• ASRE uses constrained action sampling and action distribution regularization.

Main Results:

• ASRE effectively constrains action sampling in sparse action environments.
• ASRE outperforms baseline algorithms in tasks with limited action execution.
• Demonstrated broad applicability and performance improvement in Atari games.

Conclusions:

• ASRE provides a robust solution for RL problems with sparse actions.
• The algorithm shows theoretical convergence and practical effectiveness.
• ASRE enhances RL agent performance in budget-constrained decision-making scenarios.