Control of neural systems at multiple scales using model-free, deep reinforcement learning.

Area of Science:

• Neuroscience
• Machine Learning
• Control Theory

Background:

• Practical neural control is becoming more accessible due to hardware and data collection advancements.
• Current neural control methods predominantly rely on complex, difficult-to-design model-based approaches.
• A need exists for more flexible and less complex methods in neural control.

Purpose of the Study:

• To adapt and apply a model-free reinforcement learning method, Deep Deterministic Policy Gradients (DDPG), to neural control.
• To demonstrate the efficacy of DDPG in solving complex neural control problems without explicit system dynamics modeling.
• To explore the potential of DDPG for both low-level and high-level neural dynamics.

Main Methods:

• Utilized Deep Deterministic Policy Gradients (DDPG), a model-free reinforcement learning algorithm.
• Applied DDPG to simulated models of low-level and high-level neural dynamics.
• Evaluated DDPG's performance on challenging control tasks, including oscillator synchrony and neural network trajectory control.

Main Results:

• DDPG successfully solved complex neural control problems that are intractable for current methods.
• Demonstrated the induction of global synchrony in weakly coupled oscillators.
• Showcased control of trajectories within the latent phase space of an underactuated neural network.

Conclusions:

• Model-free reinforcement learning, specifically DDPG, offers a flexible and powerful framework for neural control.
• DDPG can address complex problems in neural dynamics, surpassing limitations of existing techniques.
• Advances in reinforcement learning hold significant promise for solving fundamental problems in neural control and enabling more sophisticated real-world applications.