Sign potential-driven multiplicative optimization for robust deep reinforcement learning

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Deep Reinforcement Learning (DRL) offers solutions for complex problems in robotics, autonomous driving, and finance.
• DRL models often suffer from training instability and sensitivity, necessitating robust optimization methods.

Purpose of the Study:

• To introduce a novel momentum-based optimization approach for Deep Reinforcement Learning.
• To address limitations in existing multiplicative update methods, specifically parameter sign-flipping.

Main Methods:

• Developed a momentum-based optimizer incorporating a sign-change mechanism inspired by spiking neural networks.
• The proposed method allows parameters to change signs, enhancing multiplicative updates.

Main Results:

• The novel optimizer demonstrated effectiveness in accelerating learning and improving robustness during DRL agent training.
• Experimental evaluations across various tasks confirmed the proposed method's benefits for DRL training.

Conclusions:

• The proposed optimization approach significantly enhances the stability and efficiency of Deep Reinforcement Learning.
• This method provides a robust solution for training DRL agents, overcoming limitations of current techniques.