Prediction-based variable selection for component-wise gradient boosting.

Area of Science:

• Statistics
• Machine Learning
• Computational Statistics

Background:

• Model-based component-wise gradient boosting is widely used for data-driven variable selection.
• Existing modifications primarily address stopping criteria, not the core variable selection mechanism.
• There is a need for improved prediction and selection qualities in gradient boosting algorithms.

Purpose of the Study:

• To investigate and implement novel prediction-based variable selection mechanisms for model-based component-wise gradient boosting.
• To evaluate the efficacy of Akaike's Information Criterion (AIC) and cross-validation for variable selection.
• To assess the impact of these methods on both variable selection properties and predictive performance.

Main Methods:

• Implementation of Akaike's Information Criterion (AIC) for variable selection.
• Development and application of a component-wise test error selection rule using cross-validation.
• Evaluation using Generalized Linear Models (GLMs) within the gradient boosting framework.
• Extensive simulation studies and a real-world data application.

Main Results:

• The proposed prediction-based methods demonstrated improved variable selection properties compared to existing approaches.
• A reduction in prediction error was observed in a real-world application involving COVID-19 incidence rates.
• The cross-validation approach showed particular promise in enhancing model performance.

Conclusions:

• Prediction-based variable selection mechanisms offer a significant advancement for model-based component-wise gradient boosting.
• The integration of AIC and cross-validation can lead to more accurate and parsimonious models.
• These enhanced methods have practical implications for improving statistical modeling and prediction in various fields.