A Lagrangian-inspired force-based loss for deep metric learning
Related Experiment Videos
Byung Do Cheon1, Jeongmin Yu2, Jungeun Kim2
1Graduate School of Information, Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea.
Abstract:
Deep metric learning objectives typically define desired embedding distributions while leaving interaction directions and magnitudes implicit. We introduce a Lagrangian-inspired force-first framework prescribing and integrating conservative embedding-space interactions into a scalar objective. Under a fixed interaction configuration, the objective's negative gradient recovers the prescribed field. The framework identifies and quantifies three force-design principles: intrinsic hard mining, margin control through anchor-adaptive interaction ranges, and cardinality-weighted force balancing. Their force-based realization defines the potential energy metric loss, while an annulus-scaled projection preserves radial variation and bounds embedding norms. In frozen-encoder linear evaluation, our loss achieved the highest observed top-1 accuracies among compared objectives: 96.04%, 77.61%, and 85.94% on CIFAR-10, CIFAR-100, and ImageNet-100, respectively, with gains over the supervised contrastive loss reaching 2.88 percentage points. On long-tailed CIFAR-100 (imbalance ratio 200), accuracy was 55.54% versus 33.43% for the supervised contrastive loss, a 22.11-percentage-point gain. At this imbalance, the scale-independent ratio of within-class dispersion to between-centroid distance was lower overall (0.7387 versus 0.8372) and among the ten rarest classes (1.0607 versus 1.3000), indicating better-preserved class-level organization under cardinality skew. In class-disjoint fine-grained retrieval, our loss achieved the highest observed Recall@K at every evaluated depth on CUB-200-2011 and Cars196, with a minimum 4.07-percentage-point margin over the corresponding second-best result, demonstrating effective transfer to held-out categories. The annulus-scaled projection improved all five evaluated Euclidean metric-learning objectives over ℓ2 normalization by 1.91-7.16 percentage points. These results establish force-first objective construction as a principled and empirically effective route from desired embedding interactions to metric-learning objectives.
Related Concept Videos
Reducing Line Loss
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss in...
Lagrange Multipliers: One Constraint
Lagrange Multipliers: Two Constraints
Lagrange Multipliers: Problem Solving
Maximizing the Directional Derivative
Two-Dimensional Force System