Continuous Action Reinforcement Learning From a Mixture of Interpretable Experts.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Reinforcement learning (RL) excels at complex tasks using non-linear models but often results in opaque 'black-box' policies.
• Deploying 'black-box' RL policies in real-world scenarios raises transparency and interpretability concerns.

Purpose of the Study:

• To develop a reinforcement learning (RL) framework that generates transparent, human-readable policies.
• To address the limitations of 'black-box' policies in real-world RL applications.

Main Methods:

• Proposes a policy iteration scheme that uses complex function approximators for value prediction.
• Constrains the policy to a hierarchical structure composed of interpretable 'experts'.
• Experts select actions based on proximity to prototypical states, derived from trajectory data, addressing non-differentiable selection challenges.

Main Results:

• The proposed algorithm achieves performance comparable to neural network policies on continuous action deep RL benchmarks.
• Learned policies are significantly more interpretable and amenable to human inspection than standard neural network or linear-in-feature policies.

Conclusions:

• The developed method offers a viable alternative to 'black-box' policies in RL.
• This approach enhances the practical applicability of RL in real-world systems by providing interpretable decision-making processes.