Nonlinear Feature Selection Neural Network via Structured Sparse Regularization.

Area of Science:

• Machine Learning
• Data Science
• Computational Statistics

Background:

• High-dimensional data presents challenges due to noise and redundancy.
• Existing feature selection methods often fail to capture nonlinear relationships between data and labels.
• Linear models degrade performance in real-world applications with complex data structures.

Purpose of the Study:

• To propose a novel nonlinear feature selection method for high-dimensional datasets.
• To effectively capture and leverage the nonlinear structure inherent in data.
• To improve the selection of relevant and discriminative features for downstream tasks.

Main Methods:

• Utilizes a neural network to learn the nonlinear data structure.
• Employs a cross-entropy loss function for training.
• Applies structured sparsity regularization (l2,p-norm) to the input layer weights.
• Optimizes the model using gradient descent.

Main Results:

• Achieves a structural sparse weights matrix through nonlinear learning.
• Demonstrates superior performance compared to existing methods.
• Validated effectiveness on both synthetic and real-world datasets.

Conclusions:

• The proposed nonlinear feature selection model is effective and superior for high-dimensional data.
• Capturing nonlinear structures is crucial for robust feature selection.
• This method enhances feature selection by addressing limitations of linear approaches.