Unsupervised Discriminative Feature Selection With $\ell _{2,0}$ℓ2,0-Norm Constrained Sparse Projection

Area of Science:

• Machine Learning
• Data Science
• Computer Vision

Background:

• Feature selection is crucial for data analysis and machine learning.
• Existing sparsity-based methods using $\ell _{2,p}$-norm ($0 \lt p \leq 1$) often yield suboptimal feature subsets and require extensive parameter tuning.
• Optimizing the non-convex $\ell _{2,0}$-norm constrained problem for feature selection is an open challenge, with existing algorithms lacking guaranteed global convergence or relying on specific data assumptions.

Purpose of the Study:

• To propose an unsupervised discriminative feature selection method addressing the limitations of existing approaches.
• To introduce a novel method, Sparse Projection with $\ell _{2,0}$-norm Constraint (SPDFS), for effective unsupervised feature selection.
• To develop robust optimization strategies for the NP-hard $\ell _{2,0}$-norm constrained problem.

Main Methods:

• The proposed SPDFS method jointly learns fuzzy membership and $\ell _{2,0}$-norm constrained projection for feature-wise sparsity.
• Two optimization strategies are employed: a non-iterative algorithm for a special case guaranteeing global optimality and an iterative algorithm with ascent property for the general case.
• The method builds upon the principles of supervised linear discriminant analysis adapted for unsupervised learning.

Main Results:

• Experimental results on synthetic and real-world datasets demonstrate the effectiveness of SPDFS.
• The proposed method outperforms several state-of-the-art feature selection techniques.
• SPDFS shows superior performance in unsupervised tasks such as data clustering and text classification.

Conclusions:

• SPDFS offers a powerful and effective solution for unsupervised discriminative feature selection.
• The developed optimization strategies successfully address the NP-hard nature of $\ell _{2,0}$-norm optimization.
• The method's superiority in clustering and text classification validates its practical applicability.