DTC: Deep Tracking Control

Area of Science:

• Robotics
• Control Systems
• Machine Learning

Background:

• Legged locomotion presents complex control challenges, traditionally addressed by model-based methods like trajectory optimization.
• Model-based methods offer accuracy and insight but struggle with model inaccuracies and assumption violations.
• Simulation-based reinforcement learning (RL) excels in robustness but faces difficulties with sparse rewards in challenging environments.

Purpose of the Study:

• To develop a hybrid control architecture combining model-based planning and deep neural network-based RL.
• To achieve enhanced robustness, precise foot-placement, and improved terrain generalization for legged robots.
• To overcome limitations of purely model-based or data-driven approaches in complex locomotion tasks.

Main Methods:

• A hybrid control architecture integrating a model-based planner and a deep neural network policy.
• The model-based planner generates reference motions and optimized footholds during training.
• A deep neural network policy is trained in simulation to track these footholds, enhancing robustness and accuracy.

Main Results:

• The hybrid approach demonstrated high accuracy on sparse terrains where purely data-driven methods fail.
• Superior robustness was observed on slippery or deformable ground compared to traditional model-based methods.
• The proposed tracking controller generalized effectively across various unseen trajectory optimization methods.

Conclusions:

• The hybrid control architecture successfully combines the predictive power of online planning with the robustness of offline learning.
• This unified approach offers significant improvements in legged robot control for real-world applications.
• The method provides a promising direction for developing more capable and adaptable legged locomotion systems.