LassoNet: Neural Networks with Feature Sparsity.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Data Science

Background:

• Interpretability of neural networks is a growing research area.
• Traditional feature selection methods like Lasso (ℓ1-regularization) are limited to linear models.
• Existing neural network approaches often lack integrated feature selection.

Purpose of the Study:

• To introduce LassoNet, a novel neural network framework for global feature selection.
• To enable neural networks to learn with a reduced, relevant subset of features.
• To integrate feature selection directly into the neural network training process.

Main Methods:

• LassoNet achieves feature sparsity by conditioning feature participation in hidden units on the activity of their linear representations.
• A modified objective function with constraints integrates feature selection with parameter learning.
• The method employs projected proximal gradient descent and generalizes to deep networks.

Main Results:

• LassoNet delivers a regularization path of solutions, offering varying degrees of feature sparsity.
• Experiments on real and simulated data demonstrate significant performance improvements over state-of-the-art methods.
• The framework provides superior feature selection and regression performance.

Conclusions:

• LassoNet offers an effective and integrated approach to global feature selection in neural networks.
• The method enhances model interpretability and predictive accuracy.
• LassoNet is easily implementable, requiring minimal code modifications to standard neural networks.