Structure-Preserving Imitation Learning With Delayed Reward: An Evaluation Within the RoboCup Soccer 2D Simulation...

Area of Science:

• Robotics
• Artificial Intelligence
• Machine Learning

Background:

• Fully autonomous soccer teams in RoboCup require sophisticated control architectures.
• Existing behavioral modules in teams like Gliders2d are often evolved with human expert input.

Purpose of the Study:

• To design and evaluate a neural network architecture for improving autonomous soccer team performance.
• To assess the feasibility of replacing existing modules with a neural network approach.
• To investigate the impact of preserving expert-designed structures on neural network performance.

Main Methods:

• Utilized a deep Q-network for action determination and a deep neural network for parameter learning.
• Integrated the neural network into the Gliders2d RoboCup team.
• Introduced a delayed reward signal to aid the training process.

Main Results:

• The neural network-based architecture proved feasible for module replacement in a complex team.
• The addition of a delayed reward signal facilitated training.
• Performance was evaluated against a known benchmark.

Conclusions:

• Neural network architectures, specifically deep Q-networks, can effectively enhance autonomous robotic soccer performance.
• The integration of expert-designed behavioral structures can be maintained within neural network solutions.
• The proposed methods offer a viable path for improving AI agents in simulated environments.