MIP-BOOST: Efficient and Effective L ₀ Feature Selection for Linear Regression.

Area of Science:

• Mathematical Optimization
• Statistical Learning
• Computational Statistics

Background:

• Mixed Integer Programming (MIP) is an emerging feature selection technique.
• Standard MIP methods face challenges in parameter tuning and handling feature collinearity.
• Existing regularization methods are popular but may not be optimal for all regression scenarios.

Purpose of the Study:

• To propose MIP-BOOST, an improved MIP-based L0 feature selection method.
• To reduce the computational burden of parameter tuning in MIP feature selection.
• To enhance performance in regression with collinear features and varying signal strengths.

Main Methods:

• Revision of standard Mixed Integer Programming feature selection.
• Development of strategies to reduce computational burden for the sparsity bound parameter.
• Integration of cross-validation tuning and exact optimization of Mixed Integer Programs.
• Implementation of three synergistic proposals for improved efficiency and effectiveness.

Main Results:

• MIP-BOOST offers a more efficient and effective L0 feature selection method.
• Reduced computational burden in tuning the sparsity bound parameter.
• Improved performance with feature collinearity and diverse signal characteristics.
• Demonstrated computational viability for realistic applications.

Conclusions:

• MIP-BOOST provides a robust and efficient alternative for L0 feature selection.
• The method is suitable for large-scale and complex regression problems.
• Enhanced performance and reduced tuning complexity make MIP-BOOST practical for real-world applications.