Feature Selection Using a Neural Network With Group Lasso Regularization and Controlled Redundancy.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Data Science

Background:

• Feature selection (FS) is crucial for dimensionality reduction and improving model performance.
• Controlling redundancy among selected features remains a significant challenge in existing FS methods.
• Neural network-based approaches offer potential for sophisticated feature selection.

Purpose of the Study:

• To propose a novel neural network-based feature selection (FS) scheme.
• To integrate Group Lasso and a new redundancy-control penalty into a single objective function.
• To enable control over the level of redundancy in selected features.

Main Methods:

• The proposed scheme utilizes a neural network architecture with L2,1-norm penalties on the weight matrix.
• Group Lasso penalty promotes group-wise sparsity, while a dependence-based penalty controls redundancy.
• A smoothing technique is employed to handle nonsmooth penalty terms, ensuring algorithmic stability.

Main Results:

• The proposed algorithm's monotonicity and convergence are theoretically proven.
• Extensive experiments on artificial and real datasets demonstrate the FS scheme's effectiveness.
• Empirical results confirm the ability to control feature redundancy and align with theoretical predictions.

Conclusions:

• The developed neural network-based FS scheme effectively reduces feature redundancy.
• The integrated penalty approach offers a robust method for selecting informative and non-redundant features.
• The findings suggest practical applicability in various machine learning tasks requiring interpretable feature subsets.