A Semantic-Enhanced Multi-Source Fusion Localization Method for GNSS-Degraded Environments

In complex urban environments, Global Navigation Satellite System (GNSS) signals are easily affected by building blockage and multipath effects, which may degrade positioning quality or even cause GNSS denial. As a result, conventional integrated navigation systems suffer from accumulated errors due to insufficient global constraints. To address this problem, a multi-source integrated positioning method incorporating semantic information is proposed. Fixed traffic lights are selected as semantic landmarks, and an object detection network is used to extract the center pixel coordinates and detection confidence of the landmarks. Then, by combining depth information, camera pose, and the prior global coordinates of fixed semantic landmarks, a semantic target inversion model is established to transform two-dimensional image information into three-dimensional position estimates in the world coordinate system. Semantic factors are further constructed and incorporated into backend factor graph optimization. To determine the weighting of semantic factors, the influences of pixel localization error, depth estimation error, camera pose error, and prior coordinate error of fixed semantic landmarks on semantic observations are analyzed, and a noise covariance model for semantic factors is established. Finally, an unmanned ground vehicle experimental platform is built to validate and analyze the proposed factor graph algorithm. The experimental results show that, under GNSS-degraded conditions, the algorithm with semantic factors can provide supplementary global constraints for the system and effectively suppress accumulated positioning errors. In Experiment 1, compared with the algorithm without semantic factors, the maximum absolute trajectory error is reduced by 46.26%. To further verify the applicability of the proposed method in more complex scenarios, Experiment 2 is conducted on a longer route with multiple semantic landmarks and a more severe GNSS-degraded interval. The results show that the proposed method reduces the maximum APE from 6.5432 m to 3.4778 m, corresponding to a reduction of approximately 46.85%. These results demonstrate that the proposed semantic factor can improve the robustness of multi-source fusion localization in GNSS-degraded environments.