Simultaneously Learning Neighborship and Projection Matrix for Supervised Dimensionality Reduction.

Area of Science:

• Machine Learning
• Data Science
• Computer Vision

Background:

• Dimensionality reduction methods often rely on pre-defined neighborhood information in high-dimensional spaces.
• The curse of dimensionality complicates accurate similarity measurement and neighbor selection.
• Existing methods may yield suboptimal projection matrices due to inaccurate neighborhood assumptions.

Purpose of the Study:

• To develop a dimensionality reduction technique that jointly optimizes sample similarities and projection matrices.
• To address the limitations of fixed neighborhood assumptions in high-dimensional data.
• To improve classification and generalization performance by learning adaptive neighborhood structures.

Main Methods:

• Proposes a unified objective function to simultaneously learn optimal similarity and projection matrices.
• Models similarities and neighbors as variables within a low-dimensional space.
• Incorporates non-negativity and sum-to-one constraints on similarity matrices.
• Treats the regularization parameter as an optimizable variable, providing adaptive regularization.

Main Results:

• Demonstrates the effectiveness of the proposed method on benchmark datasets (YALE B, COIL-100, MNIST).
• The adaptive regularization parameter shows an intuitive meaning related to low-dimensional neighbors.
• Achieves improved performance compared to methods relying on fixed neighborhood information.

Conclusions:

• The proposed method offers a robust approach to dimensionality reduction by adaptively learning neighborhood structures.
• Joint optimization of similarities and projection matrices enhances classification and generalization capabilities.
• The adaptive regularization parameter provides a more principled way to tune the model.