Stable variable ranking and selection in regularized logistic regression for severely imbalanced big binary data.

Area of Science:

• Statistics
• Machine Learning
• Data Science

Background:

• High-dimensional datasets often exhibit severe class-imbalance and correlated covariates.
• Accurate variable selection is crucial for reliable model building in such data.
• Existing methods struggle with the combined challenges of class-imbalance and high dimensionality.

Purpose of the Study:

• To develop a novel covariate ranking and selection algorithm for regularized ordinary logistic regression (OLR) models.
• To address severe class-imbalance and high dimensionality in datasets with correlated covariates.
• To enhance the stability and accuracy of variable selection.

Main Methods:

• Response-based subsampling to resolve class-imbalance and stabilize variable selection.
• Ensemble of regularized OLR models fitted to subsampled, balanced datasets.
• Incorporation of regularization techniques: Lasso, adaptive Lasso (adaLasso), and ridge regression.

Main Results:

• The proposed algorithm demonstrates robustness against severe class-imbalance and highly correlated covariates.
• Consistent and stable variable selection with a very low false discovery rate.
• Effective performance across both hard- and soft-shrinkage regularization methods.

Conclusions:

• The developed framework provides a robust approach for variable selection in severely imbalanced, high-dimensional binary data.
• The methodology is versatile and applicable to various regularization techniques.
• Successful illustration on a large-scale wildland fire occurrence dataset confirms practical utility.