Approximate Orthogonal Sparse Embedding for Dimensionality Reduction.

Area of Science:

• Machine Learning
• Dimensionality Reduction
• Manifold Learning

Background:

• Locally Linear Embedding (LLE) is a prominent manifold learning technique.
• Orthogonal Neighborhood Preserving Projection (ONPP) is a linear extension of LLE widely used for dimensionality reduction.

Purpose of the Study:

• To propose a unified sparse learning framework that extends LLE-based methods to sparse scenarios.
• To establish theoretical connections between ONPP and the novel sparse linear embedding.
• To introduce sparse kernel embedding for nonlinear sparse feature extraction.

Main Methods:

• Introduced sparsity via L1-norm learning into the LLE framework.
• Developed a general model for sparse linear and nonlinear (kernel) subspace learning.
• Utilized modified elastic net and singular value decomposition for computing sparse embeddings.

Main Results:

• Demonstrated theoretical connections between ONPP and the proposed sparse linear embedding.
• Showcased the proposed model as a general framework for sparse subspace learning.
• Achieved superior performance compared to existing algorithms in experiments, particularly with small sample sizes.

Conclusions:

• The proposed sparse learning framework effectively extends LLE-based methods for dimensionality reduction.
• The novel sparse linear and kernel embeddings offer robust feature extraction, outperforming existing methods.
• The framework demonstrates significant advantages in scenarios with limited data points.