Sparse-Input Neural Network using Group Concave Regularization

Area of Science:

• Machine Learning
• Computational Statistics
• Artificial Intelligence

Background:

• Simultaneous feature selection and non-linear function estimation is difficult, especially with high-dimensional data.
• Existing methods like group LASSO in neural networks may select irrelevant variables due to over-shrinkage.

Purpose of the Study:

• To propose a novel framework for feature selection in neural networks using group concave regularization.
• To address limitations of existing methods in handling high-dimensional settings and improve model sparsity.

Main Methods:

• Developed a sparse-input neural network framework employing a concave penalty on the L2 norm of input node weights.
• Implemented a backward path-wise optimization algorithm for stable solution paths in complex optimization landscapes.

Main Results:

• The proposed framework achieves effective feature selection in both low- and high-dimensional settings.
• Theoretical analysis guarantees finite-sample performance for variable selection consistency and prediction accuracy.
• Demonstrated effectiveness across continuous, binary, and time-to-event outcomes via simulations and real-world data.

Conclusions:

• The novel sparse-input neural network framework with group concave regularization offers a robust solution for feature selection.
• The method enhances prediction accuracy and ensures variable selection consistency, outperforming traditional approaches in complex modeling scenarios.