Emergence of integrated behaviors through direct optimization for homeostasis

Area of Science:

• Computational neuroscience
• Robotics
• Artificial intelligence

Background:

• Homeostasis is crucial for organisms to maintain stable internal physiological states.
• Homeostatic reinforcement learning (RL) explains animal behavior by integrating internal dynamics with environmental factors.
• Existing homeostatic RL models are limited in scale due to complex coupled dynamics.

Purpose of the Study:

• To develop a scalable homeostatic RL method using deep RL.
• To identify optimal reward functions for homeostatic learning.
• To create benchmark environments for evaluating homeostatic RL agents.

Main Methods:

• Implemented scaled-up homeostatic RL using deep reinforcement learning.
• Evaluated various reward definitions, identifying the drive function difference as optimal.
• Developed two benchmark environments for training and analyzing agents.
• Extended the method to incorporate visual input using deep convolutional neural networks.

Main Results:

• Trained agents exhibited adaptive behaviors influenced by their internal physiological states.
• The drive function difference reward definition yielded superior performance.
• Agents demonstrated visual saliency and multimodal internal representations linked to survival.

Conclusions:

• The proposed deep RL approach enables scaled-up homeostatic learning for autonomous agents.
• The findings provide a foundation for developing robots with integrated survival behaviors.
• The study highlights the importance of internal state-driven behavior and visual processing for agent survival.