Cross-Validated Bagged Learning.

Area of Science:

• Statistical Learning
• Machine Learning
• High-Dimensional Data Analysis

Background:

• Estimating high-dimensional parameters from data is crucial for many applications, including predicting outcomes based on input variables.
• Bootstrap aggregating (bagging) is a technique used to reduce estimator variance, often in conjunction with cross-validation for parameter tuning.
• Current practices often involve internal cross-validation within bagging, which may not achieve the optimal bias-variance trade-off.

Purpose of the Study:

• To propose and evaluate a novel cross-validated bagging method for parameter estimation.
• To demonstrate that external application of cross-validation to bagged estimators optimizes the bias-variance trade-off.
• To compare the performance of the new method against existing bagging strategies.

Main Methods:

• Developing a cross-validated bagging approach where cross-validation is applied externally to bagged estimators.
• Comparing the proposed method with two baseline approaches: bagging of cross-validated estimators and bagging of non-cross-validated estimators.
• Utilizing three simulation studies to assess the effectiveness of each method.

Main Results:

• The new cross-validated bagging method demonstrates superior performance in achieving the desired bias-variance trade-off.
• Simulations indicate that external cross-validation leads to more accurate parameter estimation compared to internal cross-validation within bagging.
• The proposed method effectively addresses challenges associated with high-dimensional data and complex distributions.

Conclusions:

• The external application of cross-validation to bagged estimators is essential for optimal bias-variance trade-off in high-dimensional settings.
• The novel cross-validated bagging method offers a more effective strategy for parameter estimation than current common practices.
• This approach enhances the reliability and accuracy of statistical learning models in prediction tasks.