Double Sparse Deep Reinforcement Learning via Multilayer Sparse Coding and Nonconvex Regularized Pruning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Control Systems

Background:

• Deep reinforcement learning (DRL) performance is hindered by model interference and redundant parameters.
• Effective data representation is crucial for DRL in decision-making tasks.

Purpose of the Study:

• To develop a double sparse DRL algorithm to improve control performance.
• To reduce interference and unnecessary parameters in DRL models.

Main Methods:

• Proposed a multilayer sparse-coding-structural network for deep sparse representation.
• Employed a nonconvex log regularizer for efficient weight pruning.
• Developed a double sparse DRL algorithm integrating representation learning and pruning.

Main Results:

• The proposed method demonstrated superior performance in five benchmark environments compared to existing DRL algorithms.
• Achieved significant improvements in tasks like Mountain Car (140.81 steps) and Catcher (286.08 scores).
• Reduced model parameters by over 80% while maintaining or improving performance.

Conclusions:

• The double sparse DRL effectively learns deep sparse representations, mitigating interference.
• The algorithm successfully removes redundant weights, leading to enhanced control performance and efficiency.
• This approach offers a promising direction for developing more robust and efficient DRL systems.