Hidden Brain State-based Internal Evaluation Using Kernel Inverse Reinforcement Learning in Brain-machine Interfaces.

Area of Science:

• Neuroscience
• Machine Learning
• Biomedical Engineering

Background:

• Reinforcement learning (RL)-based brain-machine interfaces (BMIs) are crucial for neural prosthetics.
• Effective reward signal design is vital for RL-based decoder efficiency.
• Current reward methods struggle to accurately capture internal subject evaluations.

Purpose of the Study:

• To develop a Hidden Brain State-based Kernel Inverse Reinforcement Learning (HBS-KIRL) method.
• To accurately infer subject-specific internal evaluations from neural activity.
• To enhance the performance of RL-based BMI decoders.

Main Methods:

• Utilized a state-space model to project neural states into a low-dimensional hidden brain state space.
• Applied kernel methods for faster convergence in reproducing kernel Hilbert space (RKHS).
• Tested HBS-KIRL on rat medial prefrontal cortex (mPFC) data during a two-lever-discrimination task.

Main Results:

• HBS-KIRL provided stable and accurate state-value distribution estimation correlated with behavior.
• Decoders trained with HBS-KIRL demonstrated consistent and superior decoding performance over time.
• Compared to naïve IRL and PCA-based IRL, HBS-KIRL showed improved state-value estimation.

Conclusions:

• HBS-KIRL effectively infers subject-specific internal evaluations from neural data.
• This method significantly improves BMI decoder training and performance.
• HBS-KIRL holds potential for advancing RL-based BMI applications.