Minimax sparse logistic regression for very high-dimensional feature selection.

Area of Science:

• Machine Learning
• Statistical Modeling
• Bioinformatics

Background:

• Logistic Regression (LR) is widely used due to its strong convexity and probabilistic nature.
• Feature selection is crucial in high-dimensional problems for model interpretability and robust predictions.
• Existing sparse LR models struggle with efficiency and unbiased solutions for very high-dimensional datasets.

Purpose of the Study:

• To propose a novel minimax sparse logistic regression model for effective feature selection.
• To address the challenges of efficiently obtaining unbiased sparse solutions in very high-dimensional settings.
• To enhance prediction accuracy and model interpretability through targeted feature selection.

Main Methods:

• A new minimax sparse logistic regression model is developed.
• The model is efficiently solved using a cutting plane algorithm.
• Nonsmooth minimax subproblems are tackled with a smoothing coordinate descent method.

Main Results:

• The proposed method demonstrates improved prediction accuracy for a given number of selected features.
• It exhibits competitive or superior scalability on very high-dimensional problems compared to baseline methods like l1-regularized LR.
• Numerical stability and convergence rates of the smoothing coordinate descent method were analyzed.

Conclusions:

• The minimax sparse LR model offers an efficient and effective approach for feature selection in high-dimensional data.
• The developed method provides a valuable tool for applications requiring interpretable and robust predictive models.
• This work advances the state-of-the-art in sparse logistic regression for large-scale feature selection problems.