Cross-Detector Visual Localization with Coplanarity Constraints for Indoor Environments

Most visual localization (VL) methods typically assume that keypoints in the query image are detected with the same algorithm as those stored in the reference map. This poses a serious limitation, as new and better detectors may progressively appear, and we would like to ensure the interoperability and coexistence of cameras with heterogeneous detectors in a single map representation. While rebuilding the map with new detectors might seem a solution, it is often impractical, as original images may be unavailable or restricted due to data privacy constraints. In this paper, we address this challenge with two main contributions. First, we introduce and formalize the problem of cross-detector VL, in which the inherent spatial discrepancies between keypoints from different detectors hinder the process of establishing correct correspondences when relying strictly on the similarity of descriptors for matching. Second, we propose CoplaMatch, the first approach to solve this problem by relaxing strict descriptor similarity and imposing geometric coplanarity constraints. The latter is achieved by leveraging 2D homographies between groups of query and map keypoints. This process involves segmenting planar patches, which is performed offline once for the map, and also in the query image, which adds an extra computational overhead to the VL process, although we demonstrated in our experiments that this does not hinder the online applicability. We extensively validate our proposal through experiments in indoor environments using real-world datasets, demonstrating its effectiveness against two state-of-the-art methods by enabling accurate localization in cross-detector scenarios. Additionally, our work validates the feasibility of cross-detector VL and opens a new direction for the long-term usability of feature-based maps.