Autonomous learning of features for control: Experiments with embodied and situated agents

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Continuous control optimization is crucial for intelligent agents.
• Enhancing reinforcement learning (RL) and evolutionary algorithms (EA) is an active research area.
• Self-supervised learning offers a promising avenue for feature extraction.

Purpose of the Study:

• To introduce a novel method for continuous control optimization.
• To enable parallel training of feature extraction and control networks.
• To investigate the benefits of self-supervised feature extraction in continuous control tasks.

Main Methods:

• Developed a method for concurrent training of feature extraction and control networks.
• Employed self-supervision for training the feature extraction network.
• Evaluated the approach on continuous control problems, including those with egocentric observations.
• Compared sequence-to-sequence learning against alternative feature extraction methods.

Main Results:

• Parallel training of feature and control networks is critical for agents using egocentric observations.
• Feature extraction provides benefits even in problems where dimensionality reduction is not the primary goal.
• Sequence-to-sequence learning demonstrated superior performance compared to other considered feature extraction techniques.

Conclusions:

• The proposed method enhances the efficacy of evolutionary and reinforcement learning algorithms.
• Self-supervised feature extraction, particularly with sequence-to-sequence models, offers significant advantages in continuous control.
• The parallel training strategy is vital for agents relying on egocentric viewpoints.