Neuroevolutionary reinforcement learning for generalized control of simulated helicopters

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Generalized simulated helicopter hovering is a complex challenge problem for reinforcement learning.
• Neuroevolution is suitable for high-dimensional state and action spaces but faces challenges in efficient exploration and online learning.

Purpose of the Study:

• To present an extended case study applying neuroevolution to generalized simulated helicopter hovering.
• To propose and evaluate methods for increasingly challenging helicopter hovering tasks.
• To analyze the effectiveness of neuroevolution for online reinforcement learning.

Main Methods:

• Application of neuroevolutionary algorithms.
• Evaluation of several methods across three variations of the helicopter hovering task.
• Incorporation of recent advances in efficient resampling techniques.

Main Results:

• Neuroevolution effectively solves complex online reinforcement learning tasks like helicopter hovering.
• Neuroevolution excels at discovering hovering policies but not at learning helicopter models.
• Model-based approaches require domain expertise for effective model representation.

Conclusions:

• Neuroevolution is a viable approach for complex online reinforcement learning problems.
• Efficient resampling advances neuroevolution's capability for generalized reinforcement learning.
• Model-based reinforcement learning for helicopter hovering is limited by model learning difficulties.