Dimensionality Reduction Using Similarity-Induced Embeddings.

Area of Science:

• Machine Learning
• Data Science
• Computational Statistics

Background:

• Traditional dimensionality reduction (DR) methods often rely on second-order statistics, leading to limitations like sensitivity to outliers and the need for complex regularizers.
• Existing DR techniques can be suboptimal when dealing with complex data distributions or when outliers are present.

Purpose of the Study:

• To introduce a novel dimensionality reduction (DR) framework, the Similarity Embedding Framework (SEF), that utilizes similarity rather than distance.
• To demonstrate SEF's ability to overcome limitations of traditional DR methods and provide a conceptually simpler optimization approach.
• To showcase SEF's versatility in handling various DR tasks, including supervised DR and out-of-sample extensions.

Main Methods:

• The proposed Similarity Embedding Framework (SEF) models the target distribution directly using a similarity-based approach.
• New DR techniques are derived by selecting appropriate target similarity matrices within the SEF.
• The framework is evaluated on six diverse datasets, demonstrating its applicability to classical and novel DR tasks.

Main Results:

• The SEF framework successfully models target distributions using similarity, addressing limitations of distance-based DR methods.
• The framework enables supervised DR, provides out-of-sample extensions, and facilitates fast linear embeddings for complex techniques.
• Empirical evaluation on six datasets shows that SEF outperforms several existing dimensionality reduction techniques.

Conclusions:

• The Similarity Embedding Framework (SEF) offers a powerful and flexible alternative to traditional distance-based dimensionality reduction methods.
• SEF's similarity-based approach simplifies optimization targets and enhances robustness against outliers.
• The framework demonstrates broad applicability and superior performance across various data science and machine learning tasks.