Deep Learning-Based Real-Time Multiple-Object Detection and Tracking from Aerial Imagery via a Flying Robot with...

Area of Science:

• Robotics and Computer Vision
• Artificial Intelligence
• Aerospace Engineering

Background:

• Increasing demand for real-time aerial target detection and tracking using drones.
• Need for efficient onboard processing capabilities on small, space-limited flying robots.

Purpose of the Study:

• To propose an effective deep learning framework for onboard drone-based target detection and tracking.
• To develop and evaluate embedded hardware systems for real-time aerial computation.
• To introduce an advanced tracking algorithm integrating deep learning with established tracking methodologies.

Main Methods:

• Development of two embedded hardware modules (Jetson TX/AGX Xavier, Intel Neural Compute Stick) for onboard computation.
• Comparative analysis of state-of-the-art deep learning object detection algorithms on embedded GPU modules.
• Implementation of an extended Simple Online and Real-Time Tracking (SORT) algorithm with a deep learning association metric (Deep SORT).
• Integration of a GPU-based guidance system for target position tracking.

Main Results:

• Demonstration of real-time onboard computation capabilities on small drones.
• Comparative metric data on frame rates and computation power for various deep learning algorithms on embedded systems.
• Successful integration of Deep SORT for effective multi-object tracking.
• Validation of the proposed algorithms through real-time experiments on a multi-rotor drone.

Conclusions:

• The proposed deep learning framework and embedded hardware enable effective real-time onboard target detection and tracking for drones.
• The integrated Deep SORT algorithm provides robust tracking of moving objects in aerial imagery.
• The system demonstrates practical applicability for aerial surveillance and guidance tasks using small unmanned aerial vehicles.