DIML: Deep Interpretable Metric Learning via Structural Matching.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Deep metric learning (DML) methods typically generate global similarity scores, limiting interpretability.
• Understanding the decision-making process in DML is crucial for reliable applications.

Purpose of the Study:

• To develop a more interpretable deep metric learning framework (DIML).
• To enhance the understanding of how models distinguish between images by analyzing local similarities.

Main Methods:

• Proposed a structural matching strategy for spatial embedding alignment via optimal matching flow.
• Introduced a multi-scale matching strategy to balance global and local similarities, reducing computational cost.
• Utilized cross-correlation within optimal transport to handle view variance and identify important image regions.
• Extended the framework for Vision Transformers (ViTs) using truncated attention rollout and partial similarity.

Main Results:

• Achieved substantial improvements on CUB200-2011, Cars196, and Stanford Online Products benchmarks.
• Demonstrated superior performance compared to existing popular metric learning methods.
• Provided enhanced interpretability by decomposing similarity into weighted local part-wise similarities.

Conclusions:

• DIML offers a significant advancement in interpretable deep metric learning.
• The framework is model-agnostic and adaptable to various backbone networks and architectures, including ViTs.
• DIML enhances both the performance and transparency of deep metric learning models.