Effective Discriminative Feature Selection With Nontrivial Solution.

Area of Science:

• Machine Learning
• Data Science
• Pattern Recognition

Background:

• Dimensionality reduction is crucial in machine learning, typically achieved through feature selection or feature transformation.
• Existing methods often treat these two approaches separately, potentially limiting their effectiveness.
• Linear Discriminant Analysis (LDA) is a popular transformation-based dimensionality reduction technique.

Purpose of the Study:

• To propose a unified feature selection method by combining LDA with sparsity regularization.
• To simultaneously select discriminative features and remove redundant ones.
• To extend the method using l2,p-norm regularization for enhanced sparsity.

Main Methods:

• Imposing row sparsity on the LDA transformation matrix using l2,1-norm regularization.
• Extending the formulation to the l2,p-norm regularized case (0 < p < 1) for improved feature selection.
• Developing an efficient algorithm to solve the l2,p-norm optimization problem, with proven convergence for 0 < p ≤ 2.

Main Results:

• The proposed method successfully integrates feature selection and transformation.
• Experimental results on diverse real-world datasets demonstrate the method's effectiveness.
• The l2,p-norm extension offers better sparsity and approximation to the feature selection problem.

Conclusions:

• The novel LDA-based feature selection method with sparsity regularization is effective.
• The developed algorithm efficiently handles the optimization problem.
• The approach provides a promising solution for dimensionality reduction and feature selection in machine learning.