Deep Metric Learning With Adaptively Composite Dynamic Constraints

In this paper, we propose a deep metric learning with adaptively composite dynamic constraints (DML-DC) method for image retrieval and clustering. Most existing deep metric learning methods impose pre-defined constraints on the training samples, which might not be optimal at all stages of training. To address this, we propose a learnable constraint generator to adaptively produce dynamic constraints to train the metric towards good generalization. We formulate the objective of deep metric learning under a proxy Collection, pair Sampling, tuple Construction, and tuple Weighting (CSCW) paradigm. For proxy collection, we progressively update a set of proxies using a cross-attention mechanism to integrate information from the current batch of samples. For pair sampling, we employ a graph neural network to model the structural relations between sample-proxy pairs to produce the preservation probabilities for each pair. Having constructed a set of tuples based on the sampled pairs, we further re-weight each training tuple to adaptively adjust its effect on the metric. We formulate the learning of the constraint generator as a meta-learning problem, where we employ an episode-based training scheme and update the generator at each iteration to adapt to the current model status. We construct each episode by sampling two subsets of disjoint labels to simulate the procedure of training and testing and use the performance of the one-gradient-updated metric on the validation subset as the meta-objective of the assessor. We conducted extensive experiments on five widely used benchmarks under two evaluation protocols to demonstrate the effectiveness of the proposed framework.