Vision-Based Multirotor Following Using Synthetic Learning Techniques

Area of Science:

• Robotics
• Computer Vision
• Artificial Intelligence

Background:

• Deep and reinforcement learning require extensive real-world data for stable convergence and generalization.
• Current research lacks robust methods to address the scarcity of real-world data using synthetic data and domain adaptation.
• Vision-based autonomous systems often struggle with generalization due to data limitations.

Purpose of the Study:

• To introduce a synthetic-learning strategy for vision-based autonomous following of a noncooperative multirotor.
• To develop a novel motion-control strategy coupling camera gimbal movement with multirotor motion.
• To validate the effectiveness of synthetic data for training deep and reinforcement learning models in robotics.

Main Methods:

• Utilized synthetic images and high-dimensional robot states for training.
• Employed deep learning for object detection and reinforcement learning for motion control.
• Implemented a coupled camera gimbal and multirotor motion control strategy.

Main Results:

• The framework successfully learned autonomous multirotor following using only synthetic data.
• Achieved stable following of a multirotor drone in both simulated and real-flight scenarios.
• Demonstrated successful real-flight following at speeds up to 0.3 m/s and simulation speeds up to 1.3 m/s.

Conclusions:

• Synthetic learning strategies are effective for deploying vision-based tasks in real-world robotic applications.
• The proposed framework overcomes the challenge of limited real-world data for training autonomous systems.
• The coupled motion control strategy enhances the performance of vision-based drone following.