An experimental evaluation of boosting methods for classification

Area of Science:

• Clinical Medicine
• Machine Learning
• Biostatistics

Background:

• Classification rule accuracy in clinical medicine is often insufficient for daily practice.
• Classification ensembles are increasingly used to improve classifier performance.
• Boosting methods are popular ensemble techniques.

Purpose of the Study:

• Compare two popular boosting methods (AdaBoost, gradient boosting) with classical statistical approaches.
• Evaluate classifier performance using clinical data and simulations.
• Assess the utility of ensemble methods versus traditional techniques in medical diagnosis.

Main Methods:

• Employed AdaBoost and gradient boosting ensembles of regression trees.
• Utilized logistic regression with fractional polynomial approach and a tree approach as competitors.
• Assessed performance using estimated misclassification rates and Brier scores on breast tumor diagnosis data and simulations.

Main Results:

• Boosting methods yielded classification rules with enhanced predictive ability.
• Boosting classifiers did not consistently outperform logistic regression.
• Logistic regression classifiers were more interpretable and easier to use than computer-intensive boosting methods.

Conclusions:

• Logistic regression remains a valuable method in medical applications, often competitive with advanced techniques.
• Optimizing the number of boosting steps may enhance the performance of boosting methods.
• Interpretability and ease of use favor logistic regression in clinical practice.