Manifold Regularized Reinforcement Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Control Theory

Background:

• Continuous Markov decision processes (CMDPs) are fundamental in reinforcement learning.
• Value function approximation in CMDPs often requires carefully engineered features.
• Existing methods may struggle with adapting to complex state-space geometries.

Purpose of the Study:

• To introduce a novel manifold regularized reinforcement learning (RL) scheme.
• To enable automatic learning of smooth feature representations for value function approximation.
• To enhance policy learning and control in CMDPs.

Main Methods:

• Utilized unsupervised manifold regularization for feature learning.
• Developed a data-driven approach adaptable to state-space geometry.
• Implemented a basis representation extension for novel samples.

Main Results:

• Demonstrated effective learning of smooth features for value function approximation.
• Showcased adaptability to the underlying state-space geometry.
• Achieved excellent performance on inverted pendulum and energy storage control tasks.

Conclusions:

• The proposed manifold regularized RL scheme effectively learns feature representations.
• The method offers a robust approach for continuous control problems.
• Simulation results validate the scheme's practical utility and performance.