A Safe Feature Elimination Rule for L₁-Regularized Logistic Regression.

Area of Science:

• Machine Learning
• Computational Statistics

Background:

• L1-regularized logistic regression (L1-LR) is widely used for classification.
• High-dimensional data presents computational challenges for L1-LR training.
• Safe screening rules accelerate L1-LR by eliminating inactive features.

Purpose of the Study:

• Introduce a novel safe feature elimination rule (SFER) for L1-LR.
• Enhance the screening ability of existing safe rules.
• Improve the training efficiency of L1-LR on high-dimensional datasets.

Main Methods:

• Developed SFER by constructing a smaller sphere region using dual L1-LR strong convexity twice.
• Incorporated multiple half-spaces into the safe region for further contraction.
• Utilized an iterative filtering framework by decomposing the safe region into "domes" for closed-form solutions.

Main Results:

• SFER demonstrates enhanced screening power compared to existing safe rules.
• The iterative filtering framework allows for closed-form solutions and avoids repeated feature scanning.
• Experiments on ten benchmark datasets show SFER outperforms existing methods in training efficiency.

Conclusions:

• SFER offers a more effective approach to feature selection for L1-LR.
• The proposed method significantly reduces computational cost in training L1-LR.
• SFER provides a promising solution for accelerating L1-LR on large-scale datasets.