Interpretability of bi-level variable selection methods.

Area of Science:

• Statistics
• Machine Learning
• Bioinformatics

Background:

• Variable selection enhances model interpretability by creating sparser models.
• Standard sparsity-focused methods may fail when predictors are correlated or contextually related.
• Bi-level selection can identify predictive members within feature groups.

Purpose of the Study:

• To investigate if bi-level variable selection techniques improve model interpretability compared to standard LASSO.
• To evaluate the performance of group exponential LASSO (GEL), sparse group LASSO (SGL), and composite minimax concave penalty (cMCP) against LASSO.
• To assess selection relevance, group consistency, and collinearity tolerance under different grouping strategies.

Main Methods:

• Applied GEL, SGL, cMCP, and LASSO for predictor selection in time-to-event, regression, and classification tasks.
• Utilized bootstrap samples from a cohort of 1001 patients.
• Compared methods using groupings based on prior knowledge, correlation, and random assignment.

Main Results:

• Bi-level selection methods consistently outperformed LASSO across all evaluated criteria.
• cMCP showed superior selection relevance.
• SGL demonstrated strong group consistency.
• GEL exhibited all-round capacity, selecting correlated and related predictors with high relevance.

Conclusions:

• Bi-level selection methods are more effective than LASSO for interpretable models with grouped or correlated variables.
• GEL is particularly recommended for its ability to jointly select related predictors while maintaining high interpretability.
• The choice of grouping strategy impacts the performance of bi-level selection methods.