Dynamic Inverse Reinforcement Learning for Feedback-driven Reward Estimation in Brain Machine Interface Tasks.

Area of Science:

• Neuroscience
• Computer Science
• Robotics

Background:

• Reinforcement learning (RL)-based brain-machine interfaces (BMIs) offer potential for individuals with paralysis.
• Effective reward signals are crucial for enhancing RL-based BMI performance.
• Inverse reinforcement learning (IRL) can infer user intent but struggles with dynamic goals in complex tasks.

Purpose of the Study:

• To develop a dynamic IRL method for estimating time-varying, feedback-driven reward signals in BMI tasks.
• To address the limitations of static IRL methods and computationally intensive dynamic IRL algorithms.
• To improve the design and decoding performance of RL-based BMIs.

Main Methods:

• Proposed a dynamic IRL method utilizing a state-observation model to infer reward values.
• Incorporated sensory feedback as external input to model reward transitions.
• Evaluated the method on a simulated multistep BMI fetch task with a time-varying reward function.

Main Results:

• The proposed dynamic IRL method accurately estimated reward values, closely matching ground truth.
• Demonstrated significant performance improvement over existing dynamic IRL methods, especially in larger state spaces (p<0.01).
• Showcased the method's effectiveness in a complex, multistep simulated BMI task.

Conclusions:

• The dynamic IRL method shows promise for improving reward estimation in RL-based BMIs.
• This approach has the potential to enhance the control capabilities and user experience for paralyzed individuals using BMIs.
• Further research into dynamic IRL is warranted for advancing BMI technology.