Confirming the statistically significant superiority of tree-based machine learning algorithms over their...

Area of Science:

• Computer Science
• Data Science
• Statistical Modeling

Background:

• Individual studies suggest tree-based machine learning (ML) algorithms outperform others.
• A lack of comprehensive statistical validation exists for these observed superiorities.
• This study aims to bridge this gap by rigorously evaluating ML algorithm performance.

Purpose of the Study:

• To statistically validate the superiority of tree-based ML algorithms over non-tree-based algorithms.
• To compare Decision Trees and Random Forests against Support Vector Machines, Logistic Regression, and k-Nearest Neighbors.
• To assess performance across diverse research contexts, including disease prediction and university ranking.

Main Methods:

• Utilized five ML algorithms: Decision Tree, Random Forest, Support Vector Machine, Logistic Regression, and k-Nearest Neighbors.
• Conducted analysis on 200 diverse, open-access datasets.
• Employed paired-sample t-tests to evaluate performance differences across four key metrics: accuracy, precision, recall, and F1 score.

Main Results:

• Paired-sample t-tests demonstrated statistically significant (p<0.001) superiority of both Decision Trees and Random Forests over the non-tree-based algorithms.
• This performance advantage was consistent across all four metrics (accuracy, precision, recall, F1 score) in both model development and testing phases.
• The superiority of tree-based algorithms was confirmed in subsets of datasets from disease prediction and university ranking contexts.

Conclusions:

• Tree-based ML algorithms, specifically Decision Trees and Random Forests, offer statistically validated superior performance compared to non-tree-based alternatives.
• The findings hold true across various ML performance measures and research domains, including specialized areas like disease prediction.
• This study provides robust evidence supporting the widespread adoption of tree-based ML algorithms for enhanced predictive modeling.