Self-tuning Optimal Regulation of Respiratory Motor Output by Hebbian Covariance Learning

The respiratory motor system is a specialized musculoskeletal system that is controlled by a small assembly of neuronal clusters in the brainstem. Its prime function is to maintain CO(2), O(2), and pH homeostasis in arterial circulation through the motor act of breathing. A longstanding dilemma is that during muscular exercise homeostatic regulation occurs automatically without any apparent feedback or feedforward signals, whereas the homeostasis is readily abolished by exogenous chemical challenge. Recently, it has been proposed that these seemingly incongruous behaviors of the respiratory controller may be a manifestation of self-tuning adaptive control. This hypothesis is supported in part by recent discoveries of various memory systems in the brainstem controller including short- and long-term potentiation and depression of synaptic transmission as well as the dramatic abolishment of homeostatic regulation in mutant mice with targeted genetic disruption of the NMDA receptors. In this paper, we propose a model of self-tuning homeostatic regulation based upon a synaptic adaptation rule-Hebbian covariance learning rule-that has been suggested to underlie many forms of learning and memory in the higher brain. We show that such an adaptation rule may be a useful neuronal substrate for reinforcement learning in optimization tasks. The model demonstrates how spontaneous oscillations and/or random-like fluctuations of neural activity may be exploited by the controller to adaptively regulate and optimize motor output. Such a self-tuning neural control paradigm, which operates without the need for any internal model of the external environment, is generally applicable to a class of steady-state optimal regulation problems with infinite time horizon. An interesting implication of the present results is that brain intelligent control can occur at a subconscious level without the need for voluntary intervention from the higher brain. Copyright 1996 Elsevier Science Ltd.