MambaAdapt: Joint Descriptor-Alignment Learning with Mamba-Transformer Fusion for Robust Long-Term VPR

Visual place recognition (VPR) is one of the important enabling technologies in localization and mapping for intelligent transportation systems (ITSs). Long-term VPR is challenging even with descriptors extracted exclusively for the discrimination of position, especially when there is a significant change in illumination conditions, seasons, viewpoints, occlusions, traffic, and/or changes in structures. Additionally, temporal adaptation is typically done after the learning of the descriptors, and adaptation goals do not directly impact the descriptor space, which can act as a conflict with existing place representations. In order to overcome this limitation, the MT-FusionNet descriptor backbone is jointly optimized with the Cross-Temporal Meta-Reweighting Recurrent Encoder (CT-MRRE) and a representation-preserving process with replay. The two steps of CT-MRRE are: residual cross-temporal alignment to correct the descriptors dislocated by the condition and suppressing unreliable descriptors in the context of the current environmental condition through recurrent reliability reweighting. Therefore, place discrimination, temporal adaptation, reliability estimation, and representation retention are all optimized in a single descriptor space. The average Recall@1 is 97.2% across the four sites, being 96.4%, 96.2%, 99.1%, and 97.0% for Oxford RobotCar, Nordland, City Center, and St Lucia, respectively. This is in line with the common four-dataset evaluation protocol, where it represents an average absolute gain of 2.8 percentage points over MT-FusionNet and of 3.7 percentage points over CerfeVPR. The complementary effect of the cross-temporal alignment, context-aware reliability reweighting and replay-supported joint optimization is revealed by the component analysis. These results show the usefulness of the combination of discriminative descriptor learning, cross-time adaptation, reliability weighting, and representation preservation in the evaluated environmental variations for long-term VPR.