Generalized Embedding Regression: A Framework for Supervised Feature Extraction.

Area of Science:

• Machine Learning
• Computer Vision
• Pattern Recognition

Background:

• Sparse discriminative projection learning is crucial for recognition tasks.
• Existing methods often lack joint optimization of embedding and sparse projection.
• Preserving global data structure and exploring class correlations are key challenges.

Purpose of the Study:

• Propose a generalized embedding regression (GER) framework for joint low-dimensional embedding and sparse projection learning.
• Introduce a novel supervised feature extraction method, jointly sparse embedding regression (JSER), based on the GER framework.
• Enhance robustness to outliers and data variations while achieving semantic interpretability.

Main Methods:

• Developed GER with a generalized orthogonal constraint, integrating label information and a rank constraint.
• Designed JSER incorporating an intrinsic graph for intraclass similarity and a penalty graph for interclass separability.
• Utilized L2,1-norm for robustness and jointly sparse projection learning, with an iterative algorithm to solve the optimization problem.

Main Results:

• Theoretical analysis shows GER's equivalence or approximation to related methods.
• JSER effectively handles interclass marginal points using penalty graph Laplacian.
• Experimental results on six datasets demonstrate JSER's competitive performance and latent properties.

Conclusions:

• The proposed GER framework provides a unified approach for embedding and sparse projection learning.
• JSER offers a powerful and interpretable supervised feature extraction method.
• The developed iterative algorithm for JSER is theoretically sound and converges effectively.