An AR 3D Tracking and Registration Method That Integrates Optical Flow Tracking and Mean Shift

Augmented reality (AR) enhances the real world scene by overlaying virtual information onto it. Vision-based 3D tracking and registration is the key technology for ensuring the fusion of virtual and real content in monocular AR systems. Existing mainstream visual tracking and registration methods are susceptible to illumination variations, motion blur, target occlusion, and dynamic background interference in complex scenarios. They also suffer from low computational efficiency, cumulative pose errors, and insufficient stability, making them difficult to deploy on low power edge devices such as embedded systems and mobile terminals. To address these issues, this paper proposes a lightweight monocular AR 3D tracking and registration method that integrates ORB-FREAK features, mismatching outlier filtering, background weighted mean shift, and template-based relocalization. The method does not rely on depth sensors or neural network inference, enabling efficient and accurate lightweight pose estimation. Specifically, we first combine the ORB (Oriented FAST and Rotated BRIEF) descriptor with the FREAK (Fast Retina Keypoint) algorithm for feature detection and initial matching. Hamming distance is used for coarse filtering of mismatched point pairs, and an ascending sort combined with an iterative sequential sampling strategy is applied to solve the optimal homography matrix, significantly improving the accuracy and efficiency of matrix estimation. Then, distance constraints among feature points are imposed on the target registration region to optimize the selection, and camera pose is computed based on the matching between 2D feature points and their corresponding 3D spatial coordinates, eliminating the error accumulation problem of conventional algorithms. Real-time feature matching is further used to correct the optical flow tracking sequence and camera pose, ensuring the continuity of the AR tracking process. Finally, a background weighted mean shift algorithm is introduced to narrow the feature detection range and suppress background interference, complemented by a template-matching relocalization module and a dynamic model update strategy, which effectively enhance the robustness of continuous tracking and registration under complex conditions. Experimental results demonstrate that, in extreme scenarios such as low light conditions, high speed motion, and occlusion, the proposed method achieves AR 3D tracking and registration success rates of 86.7%, 82.3%, and 78.5%, respectively. It exhibits superior performance in pose estimation accuracy and anti-interference capability in complex environments, with significantly reduced computational overhead. Moreover, it can achieve robust and continuous AR 3D tracking and registration on low power edge devices, effectively adapting to demanding AR application scenarios and providing reliable technical support for lightweight AR applications.