Enhanced real-time 6d pose estimation for automatic recovery of in-flight UAVs using distance-aware keypoint heatmaps

Unmanned Aerial Vehicles (UAVs) play crucial roles across various fields but face significant challenges when control is lost or securing a safe runway is not feasible. Under these circumstances, reliable UAV recovery systems are essential for safe retrieval, requiring precise and real-time localization. This paper proposes an automatic in-fight UAV recovery system using an unmanned ground vehicle when securing a safe runway is challenging such as unstructured road, and presents a practical approach for real-time six-degree-of-freedom (6-DOF) pose estimation of an in-flight UAV using monocular RGB images and deep learning-based heatmap keypoint detection. To enhance the accuracy of the pose estimation, we propose an Adaptive Sigma technique for keypoint detection, which adjusts the sigma values of the keypoint heatmap based on the distance between the camera and the UAV. Thus, the proposed method can robustly adapt to changes in the distance of UAV to improve the keypoint localization performance. By utilizing the predicted keypoints in a Perspective-n-Point algorithm, the 6-DOF pose information of the UAV can be obtained in real time. The proposed Adaptive Sigma to heatmap-based keypoint detection improves the Percentage of Correct Keypoints by up to 2.5%, with consistently high performance across various state-of-the-art backbone architectures. The proposed method qualitatively evaluated in challenging scenarios such as various altitudes, significant tilt, and motion blur.