HRL-Det: Hierarchical Reinforcement Learning for Sequential Object Detection in Aerial Imagery

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Object detection in unmanned aerial vehicle (UAV) imagery faces challenges like scale variation, dense packing, and high computational costs.
• Existing reinforcement learning (RL)-based sequential detectors struggle with discrete states, sparse rewards, and policy collapse.

Purpose of the Study:

• To propose HRL-Det, a hierarchical reinforcement learning framework to overcome limitations in RL-based object detection for UAVs.
• To enhance temporal reasoning and enable efficient active search processes for object localization.

Main Methods:

• Developed a Neural ODE-driven Continuous-Time Bellman State Evolution module for fine-grained temporal reasoning using stochastic differential equations.
• Implemented a Lyapunov-Guided Entropy-Regularized Reward Shaping mechanism for convergence-promoting dense rewards and exploration diversity.
• Utilized memory-efficient adjoint-based backpropagation for state dynamics modeling.

Main Results:

• HRL-Det achieved mAP@0.5 scores of 0.412 on VisDrone2019, 0.812 on DroneVehicle, and 0.735 on MS COCO 2017.
• Outperformed existing RL-based detectors and showed competitive accuracy against non-RL detectors.
• Required only 17.3 M parameters and an average of 6.3 search steps per object, demonstrating significant efficiency gains.

Conclusions:

• HRL-Det offers a robust and efficient solution for object detection in challenging UAV scenarios.
• The framework effectively addresses scale variation, dense packing, and computational cost issues.
• The proposed innovations in continuous-time state evolution and reward shaping contribute to improved performance and efficiency.