Kalman filter control embedded into the reinforcement learning framework.

Area of Science:

• Computational neuroscience
• Machine learning
• Control theory

Background:

• Growing interest in Kalman filter models for brain modeling.
• Need for online estimation and control capabilities in these models.
• Limitations of traditional offline optimal control methods (backward recursion).

Purpose of the Study:

• To investigate the feasibility of using Kalman filter models for online control in brain modeling.
• To develop a modified Kalman filter approach for real-time optimal control.
• To address the limitations of offline methods in dynamic brain modeling scenarios.

Main Methods:

• Modification of the linear-quadratic-gaussian Kalman filter model.
• Integration of reinforcement learning for online control estimation.
• Development of a novel learning rule for value estimation.

Main Results:

• Successful on-line estimation of optimal control using the modified Kalman filter.
• Overcoming the limitations of offline backward recursion for real-time control.
• Emergence of a Hebbian-form learning rule weighted by estimation error.

Conclusions:

• The modified Kalman filter model effectively enables online optimal control in brain modeling.
• Reinforcement learning provides a viable solution for real-time control challenges.
• The Hebbian-like learning rule offers insights into adaptive control mechanisms.