Large-Scale Metric Learning: A Voyage From Shallow to Deep

Area of Science:

• Machine Learning
• Computer Vision
• Data Science

Background:

• The Keep It Simple and Straightforward Metric Learning (KISSME) algorithm offers attractive properties but relies heavily on principal component analysis (PCA) for preprocessing.
• PCA's dependence can cause performance issues, particularly with suboptimal dimensionality settings, hindering practical application.

Purpose of the Study:

• To develop a unified formulation that integrates dimensionality reduction with metric learning, specifically addressing the limitations of the KISSME algorithm.
• To enhance the robustness and applicability of metric learning by removing the critical dependency on PCA preprocessing.

Main Methods:

• A novel joint formulation for dimensionality reduction and metric learning is proposed, building upon the KISSME framework.
• The formulation is framed as an optimization problem on the Grassmann manifold, leveraging Riemannian optimization techniques for efficient computation.
• An end-to-end deep learning approach is also derived, integrating the proposed metric learning strategy into a generic deep network architecture.

Main Results:

• The unified approach effectively combines dimensionality reduction and metric learning, mitigating the performance sensitivity associated with PCA in standard KISSME.
• Optimization on the Grassmann manifold ensures theoretical soundness and practical efficiency.
• The deep learning adaptation demonstrates the method's versatility and potential for complex, large-scale metric learning tasks.

Conclusions:

• The proposed unified formulation offers a more robust and user-friendly alternative to traditional KISSME by eliminating PCA dependency.
• Leveraging Riemannian geometry and deep learning, this work advances the field of metric learning with improved performance and broader applicability.