A simulation-based comparison of Boruta, LASSO, and Elastic Net for variable selection in logistic regression, with...

Variable selection is a central challenge in logistic regression, particularly in high-dimensional biomedical applications where correlated predictors and limited sample sizes complicate reliable identification of relevant variables. This study aims to systematically compare three widely used variable selection approaches - Boruta, LASSO, and Elastic Net - under a range of data-generating conditions and to illustrate their performance using an ovarian cancer miRNA dataset.

We conducted a simulation study across 36 logistic regression scenarios varying in sample size, dimensionality, predictor correlation, and effect magnitude. Performance was evaluated using true-positive and false-positive selection rates. In addition, all three methods were applied to a real-world serum miRNA expression dataset, and discriminative performance was assessed using the area under the receiver operating characteristic curve (AUC).

Boruta, Elastic Net, and LASSO exhibited distinct variable selection behaviors across simulation scenarios. Boruta maintained strong true-positive recovery while controlling false positives in most settings, particularly when predictors were highly correlated. Elastic Net consistently achieved high sensitivity but produced comparatively large false-positive rates. LASSO showed the most conservative behavior, recovering fewer true predictors while maintaining low false-positive rates across nearly all scenarios. In the ovarian cancer miRNA application, all three methods achieved similarly strong test-set AUC performance, despite marked differences in the size of the selected biomarker panels.

The results demonstrate clear trade-offs among the three methods. Boruta offers a favorable balance between sensitivity and specificity in highly correlated settings. Elastic Net prioritizes sensitivity at the cost of increased false discoveries, whereas LASSO provides stricter false-positive control with reduced sensitivity. These findings offer practical guidance for selecting variable selection methods in logistic regression, particularly for high-dimensional biomedical applications.