UAV's Status Is Worth Considering: A Fusion Representations Matching Method for Geo-Localization

Area of Science:

• Computer Vision
• Robotics
• Geographic Information Systems

Background:

• Visual geo-localization is critical for unmanned aerial vehicle (UAV) positioning and navigation.
• Existing methods struggle with viewpoint and appearance variations between drone and satellite imagery.
• Underestimation of external elements and multimodal interactions limits current approaches.

Purpose of the Study:

• To develop a pioneering feature fusion network (MBF) addressing differences between drone and satellite views.
• To incorporate UAV status information and exploit local scene features for improved geo-localization.
• To enhance the robustness and accuracy of visual geo-localization for UAVs.

Main Methods:

• Proposed a multimodal and bilinear pooling-inspired feature fusion network (MBF).
• Integrated UAV status information (e.g., flight height) as word embeddings concatenated with image embeddings in Transformer blocks.
• Employed hierarchical bilinear pooling (HBP) to correlate and reinforce global and local feature maps for robust representation.

Main Results:

• Achieved more discriminative deep representations for effective geo-localization.
• Reached new state-of-the-art results on benchmark datasets.
• Demonstrated significant performance boosting, with recall@1 accuracy of 89.05% (drone localization) and 93.15% (drone navigation) on University-1652.
• Showcased strong robustness across different flight heights on the SUES-200 dataset.

Conclusions:

• The proposed MBF network effectively fuses multimodal data, including UAV status, for superior visual geo-localization.
• The method significantly enhances accuracy and robustness in drone localization and navigation tasks.
• This work sets a new benchmark for visual geo-localization, particularly in challenging drone-to-satellite view scenarios.