Broad Critic Deep Actor Reinforcement Learning for Continuous Control.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Deep Reinforcement Learning (DRL) shows promise in continuous control but requires significant data and computation due to Deep Neural Networks (DNNs).
• Existing DRL methods face challenges with training efficiency and computational cost, limiting real-time applications.

Purpose of the Study:

• To propose a novel hybrid actor-critic Reinforcement Learning (RL) framework integrating Broad Learning System (BLS) with DNNs.
• To enhance the efficiency and accuracy of DRL algorithms for continuous control tasks.

Main Methods:

• Developed a hybrid framework where the critic network uses BLS for fast value estimation (ridge regression) and the actor network uses DNNs for policy gradient optimization.
• Integrated the BLS-DNN hybrid approach into Deep Deterministic Policy Gradient (DDPG), Soft Actor-Critic (SAC), and Twin Delayed DDPG (TD3) algorithms, creating BLS-augmented variants.

Main Results:

• All BLS-enhanced actor-critic algorithms demonstrated superior training efficiency compared to their original counterparts.
• The BLS-augmented DRL variants achieved higher accuracy in continuous control tasks.
• Experimental results confirmed the effectiveness of the hybrid BLS-DNN approach.

Conclusions:

• The proposed hybrid BLS-DNN framework significantly improves DRL training efficiency and accuracy.
• This approach is suitable for real-time control scenarios demanding computational efficiency and rapid adaptation.
• The generalizable hybrid design enhances existing actor-critic RL algorithms.