Z-Score Experience Replay in Off-Policy Deep Reinforcement Learning

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Reinforcement Learning

Background:

• Reinforcement learning (RL) enables agents to learn optimal policies through environmental interaction without pre-training data.
• Deep reinforcement learning (DRL) integrates deep learning with RL, offering advanced perception and decision-making for complex problems.
• Off-policy RL algorithms leverage stored experiences for exploration and exploitation, aiding in finding global optimal solutions.

Purpose of the Study:

• To enhance the utilization of experiences in off-policy reinforcement learning algorithms.
• To improve the performance and convergence speed of deep reinforcement learning.
• To address the challenge of efficient experience utilization in RL.

Main Methods:

• Proposes Z-Score Prioritized Experience Replay (Z-Score PER) as a novel technique.
• Integrates Z-Score PER into off-policy deep reinforcement learning frameworks.
• Conducts ablation experiments to validate the proposed method's effectiveness.

Main Results:

• Z-Score Prioritized Experience Replay significantly enhances the utilization of interaction experiences.
• The proposed method leads to improved performance and faster convergence in deep reinforcement learning algorithms.
• Ablation studies confirm the substantial effectiveness of Z-Score PER.

Conclusions:

• Z-Score Prioritized Experience Replay is an effective method for improving off-policy deep reinforcement learning.
• The approach enhances learning efficiency and algorithm performance.
• This work contributes to advancing the capabilities of deep reinforcement learning in solving complex sequential decision-making tasks.