Simultaneous Optimization of Discrete and Continuous Parameters Defining a Robot Morphology and Controller

The morphology and controller design of robots is often a labor-intensive task performed by experienced and intuitive engineers. Automatic robot design using machine learning is attracting increasing attention in the hope that it will reduce the design workload and result in better-performing robots. Most robots are created by joining several rigid parts and then mounting actuators and their controllers. Many studies limit the possible types of rigid parts to a finite set to reduce the computational burden. However, this not only limits the search space, but also prohibits the use of powerful optimization techniques. To find a robot closer to the global optimal design, a method that explores a richer set of robots is desirable. In this article, we propose a novel method to efficiently search for various robot designs. The method combines three different optimization methods with different characteristics. We apply proximal policy optimization (PPO) or soft actor-critic (SAC) as the controller, the REINFORCE algorithm to determine the lengths and other numerical parameters of the rigid parts, and a newly proposed method to determine the number and layout of the rigid parts and joints. Experiments with physical simulations confirm that when this method is used to handle two types of tasks-walking and manipulation-it performs better than simple combinations of existing methods. The source code and videos of our experiments are available online (https://github.com/r-koike/eagent).