AeroVerse: UAV-Agent Benchmark Suite for Simulating, Pre-training, Finetuning, and Evaluating Aerospace Embodied...

Area of Science:

• Aerospace engineering
• Artificial intelligence
• Robotics

Background:

• Existing embodied foundation models are limited to ground-level agents, neglecting UAVs.
• Research on UAV intelligent agents lacks standardized benchmarks for development and evaluation.

Purpose of the Study:

• To develop AeroVerse, a comprehensive benchmark suite for aerospace embodied foundation models.
• To facilitate simulation, pre-training, fine-tuning, and evaluation of UAV intelligence.

Main Methods:

• Introduced AeroSimulator for realistic UAV flight simulation in urban environments.
• Created AerialAgent-Ego15k (real-world) and CyberAgent-Ego500k (virtual) datasets for pre-training.
• Defined five downstream tasks and developed instruction datasets for fine-tuning.
• Developed SkyAgent-Eval, a GPT-4 based evaluation system.
• Proposed SkyAgent, a UAV-agent large model with novel mechanisms.

Main Results:

• Benchmarked ten mainstream models, revealing limitations of current visual-language models for aerospace tasks.
• SkyAgent outperformed existing methods by an average of 8.52% across four core tasks.
• Demonstrated the effectiveness of the proposed benchmark suite and SkyAgent model.

Conclusions:

• The AeroVerse benchmark suite is crucial for advancing aerospace embodied intelligence.
• SkyAgent represents a significant step forward in UAV autonomous capabilities.
• The developed resources will be released to foster community research.