End-to-end multiple object tracking in high-resolution optical sensors of drones with transformer models

Area of Science:

• Computer Vision
• Artificial Intelligence
• Robotics

Background:

• Drone aerial imaging is vital, but multi-object tracking faces accuracy and efficiency challenges.
• Traditional methods often separate detection and tracking, increasing complexity and reducing performance.
• Manual feature engineering limits conventional approaches.

Purpose of the Study:

• To propose a novel Transformer-based end-to-end framework for accurate and efficient multi-object tracking in drone imagery.
• To integrate object detection and tracking seamlessly using self-attention mechanisms.
• To enhance tracking stability and consistency through advanced feature extraction and label matching.

Main Methods:

• A Transformer-based end-to-end framework utilizing self-attention mechanisms.
• Trajectory detection label matching based on appearance, spatial, and Gaussian features.
• Cross-frame self-attention and a self-characteristics module for long-term feature extraction and semantic consistency.

Main Results:

• Significant performance improvements in multi-object tracking accuracy and efficiency.
• Demonstrated superior results on the VisDrone and UAVDT datasets.
• Successful integration of object detection and tracking into a unified pipeline.

Conclusions:

• The proposed Transformer-based framework effectively addresses limitations in drone-based multi-object tracking.
• The novel techniques for label matching and feature extraction enhance tracking robustness.
• This approach offers a promising solution for advanced aerial surveillance and analysis.