Visual attention prediction improves performance of autonomous drone racing agents

Area of Science:

• Robotics and Artificial Intelligence
• Computer Vision
• Human-Computer Interaction

Background:

• Current end-to-end autonomous flight systems lag behind human pilots in drone racing speed.
• Human pilots excel by effectively selecting task-relevant visual information, suggesting attention mechanisms are key.

Purpose of the Study:

• To investigate if neural networks imitating human eye gaze and attention can enhance autonomous drone racing performance.
• To test the hypothesis that gaze-based attention prediction improves visual information selection and decision-making in drone racing.

Main Methods:

• Collected eye gaze and flight data from 18 human drone pilots.
• Trained a visual attention prediction model using this human data.
• Employed imitation learning to train an end-to-end autonomous drone racing controller using the attention model.

Main Results:

• The attention-prediction controller achieved an 88% success rate in completing a challenging race track.
• This significantly outperformed controllers using raw RGB images (61% success) and feature tracks (55% success).
• Attention- and feature-track based models demonstrated superior generalization on unseen data compared to image-based models.

Conclusions:

• Incorporating human visual attention prediction markedly improves autonomous drone racing agent performance.
• This approach is a crucial step towards achieving fast, agile, vision-based autonomous flight that can match or exceed human capabilities.