SkyFind: A Large-Scale Benchmark Unveiling Referring Expression Comprehension for UAV.

Area of Science:

• Computer Vision
• Robotics
• Human-Computer Interaction

Background:

• Unmanned Aerial Vehicles (UAVs) are vital for human assistance, requiring effective human intention understanding for collaboration.
• Referring Expression Comprehension (REC) enables UAVs to identify human-intended targets, but current research lacks aerial scenario focus.
• Aerial REC faces unique challenges like background interference, small targets, and complex spatial relationships.

Purpose of the Study:

• To formally define UAV-based REC as a novel research problem.
• To introduce a large-scale dataset, SkyFind, for systematic study of aerial REC.
• To propose AerialREC, a baseline framework for improving REC in UAV imagery.

Main Methods:

• Formal definition of UAV-based Referring Expression Comprehension (REC).
• Introduction of SkyFind, a dataset with one million target-expression pairs for aerial REC.
• Development of AerialREC, a framework employing region searching before localization to mitigate background interference.

Main Results:

• Establishment of benchmark results on the SkyFind dataset using ten representative REC methods.
• Validation of the AerialREC framework's effectiveness in reducing background interference for UAV-based REC.
• Demonstration of the feasibility and challenges of aerial REC.

Conclusions:

• UAV-based REC is a critical but underexplored area for human-UAV collaboration.
• The SkyFind dataset and AerialREC framework provide essential resources for advancing aerial REC research.
• Future work should focus on improving REC performance in complex aerial environments.