Robust and enhanced 360° visual tracking based on dynamic gnomonic projection

Recently, 360-degree visual tracking has become increasingly important in 360-degree video processing technology. Although visual tracking technology in 2D videos has gradually matured, there is no universal method for visual tracking in 360-degree videos that can effectively address image stretching and object deformation caused by the equirectangular representation of 360-degree images. In this paper, we propose a two-part method for 360-degree visual tracking. The first part is a general method that can be integrated into any 2D visual tracking system to be applied to 360-degree videos. This part converts equirectangular images into 2D gnomonic projections, enabling the use of existing 2D tracking algorithms while mitigating image distortion. Then, building upon the UPDT algorithm, the second part integrates the general 360-degree visual tracking method with additional enhancements to improve robustness and efficiency in 360-degree visual tracking. Furthermore, when tracking performance deteriorates, it combines results from the sample set and trajectory prediction to achieve more robust and accurate tracking. In our experiments, We use two datasets in 360-degree equirectangular representation to demonstrate the effectiveness and advantages of our proposed method. Additionally, we explore the application of 360-degree visual tracking methods in editing, enabling the automatic manipulation of moving objects in 360-degree videos.