The importance of knowing when to stop. A sequential stopping rule for component-wise gradient boosting

Area of Science:

• Biostatistics
• Machine Learning in Medicine
• Computational Biology

Background:

• Component-wise boosting algorithms are widely used in biomedical regression.
• Algorithm performance heavily relies on selecting the optimal number of iterations.
• Current methods lack a fully automated strategy for determining this optimal stopping point.

Purpose of the Study:

• To introduce a fully data-driven sequential stopping rule for boosting algorithms.
• To automate the determination of the optimal iteration number during model fitting.
• To enhance the performance and reliability of boosting in biomedical applications.

Main Methods:

• A novel sequential stopping rule termed "subsampling after AIC" was developed.
• This rule integrates resampling techniques with a modified Akaike Information Criterion (AIC)-based approach.
• The method was applied to component-wise gradient boosting algorithms.

Main Results:

• The proposed "subsampling after AIC" rule demonstrated superior performance compared to previous methods.
• The rule was validated using both simulated datasets and real-world biomedical data.
• Significant improvements were observed in predicting stage II colon cancer recurrence using microarray data.

Conclusions:

• The developed sequential stopping rule effectively identifies the optimal stopping iteration for boosting algorithms.
• This automation occurs during the model fitting process, simplifying workflow.
• The rule is applicable to common loss functions, enhancing its practical utility.