Dual Uncertainty-Aware Guidance for Test-Time Visual Retrieval Under Distributional Variations

Visual retrieval systems remain highly vulnerable to test-time distribution shifts caused by corruption, degradation, and environmental changes. Unlike classification, retrieval-time adaptation must preserve ranking consistency in a similarity space, where the dominant failure mode is not only inaccurate prediction confidence but also unreliable similarity-induced pseudo-supervision. To address this issue, we propose Dual Uncertainty-Aware Guidance (DUG), a test-time adaptation framework for visual retrieval that explicitly separates three roles: feature modulation, pseudo-label reliability estimation, and uncertainty-calibrated optimization. DUG contains a Mixture of Prompts module for input-conditional feature modulation, a Max-Mean Uncertainty (MMU) estimator to reweight unreliable pseudo-matches in the similarity matrix, and a Dynamic Calibration via Evidential Uncertainty (DCU) mechanism to regulate the strength of test-time updates according to sample-wise evidential confidence. The resulting objective encourages the model to trust more reliable matches while avoiding over-aggressive adaptation on ambiguous samples. Extensive experiments on Revisited Oxford and Revisited Paris under 12 corruption types show that DUG consistently improves retrieval robustness over strong test-time adaptation baselines. We further clarify the adaptation protocol, the complementarity of MMU and DCU, and the practical scope of the method in terms of efficiency and deployment settings.