A Bayesian Approach to Policy Recognition and State Representation Learning.

Area of Science:

• Robotics
• Artificial Intelligence
• Machine Learning

Background:

• Learning from demonstration (LfD) builds behavioral models from expert examples for system control.
• Existing LfD methods often assume deterministic optimal policies or direct expert monitoring, limiting their applicability.
• These limitations hinder the use of LfD within general system identification frameworks.

Purpose of the Study:

• To develop a more general LfD framework capable of handling stochastic expert policies.
• To avoid assumptions about the optimality of expert demonstrations.
• To enable robust system identification and learn task-appropriate state representations.

Main Methods:

• A Bayesian methodology is employed to model the posterior distribution of expert controllers.
• The approach allows for arbitrary stochastic expert policies, not requiring optimality.
• Nonparametric methods are used to infer state representation complexity and learn state space partitionings.

Main Results:

• The proposed Bayesian LfD method successfully models stochastic expert behaviors.
• It effectively generalizes expert demonstrations to new situations without assuming optimality.
• The framework can infer expert state representation complexity and learn relevant state space partitions.

Conclusions:

• This work extends LfD by relaxing assumptions on expert policies, enabling broader applications.
• The Bayesian, nonparametric approach offers a powerful tool for system identification and understanding expert behavior.
• The methodology facilitates learning sophisticated state representations and task-specific state space segmentations.