A new method of concurrently visualizing states, values, and actions in reinforcement based brain machine interfaces

Area of Science:

• Neuroscience
• Machine Learning
• Robotics

Background:

• Brain-Machine Interfaces (BMIs) enable control of external devices using neural signals.
• Closed-loop BMIs involve co-adaptation between the user and the decoding agent.
• Quantifying individual contributions in such systems remains a challenge.

Purpose of the Study:

• To present the first quantification of individual subject and agent performance in a closed-loop Reinforcement Learning Brain Machine Interface (RLBMI).
• To analyze the co-adaptive process between a neural decoder and a biological subject.
• To visualize and understand the state-action value function (Q) and its influence on performance.

Main Methods:

• Implementation of a Q-learning agent using Kernel Temporal Difference (KTD)(λ) for decoding neural states.
• Utilizing a monkey's neural data to control a robotic arm's action directions.
• Visualization of states, Q-values, and actions in a 2D space to analyze performance dynamics.

Main Results:

• Demonstration of the decoder's ability to learn effective state-to-action mappings.
• Analysis of how neural states impact prediction accuracy and overall system performance.
• Identification of individual contributions to both successful and missed trials.

Conclusions:

• The proposed methodology allows for the quantification of individual performance in RLBMI systems.
• Co-adaptation dynamics between neural decoders and subjects can be effectively analyzed.
• Understanding these dynamics is crucial for improving BMI control and performance.