Robust Variable and Interaction Selection for Logistic Regression and General Index Models.

Area of Science:

• Statistics
• Machine Learning
• Data Science

Background:

• Variable selection is crucial for building robust statistical models.
• Existing methods often struggle with high-dimensional data and specific distributional assumptions.

Purpose of the Study:

• To introduce SODA, a novel forward-backward variable selection method.
• To extend SODA for application in general index models.
• To enhance robustness and performance in high-dimensional settings.

Main Methods:

• SODA employs a forward stage to add significant predictors and a backward stage to remove unimportant terms, optimizing the extended Bayesian Information Criterion (EBIC).
• The method is extended for variable selection and model fitting in general index models.
• Theoretical analysis confirms variable-selection consistency under high-dimensional conditions.

Main Results:

• SODA effectively handles high-dimensional data where predictors exceed sample size.
• It does not require joint normality assumptions, offering greater robustness than existing methods.
• SODA demonstrates superior performance with non-Gaussian design matrices in simulations and real-data applications.

Conclusions:

• SODA provides a robust and consistent variable selection approach for logistic and general index models.
• The method is particularly advantageous for high-dimensional and non-Gaussian datasets.
• SODA offers a significant advancement over existing variable selection techniques.