Grafted and Vanishing Random Subspaces.

Area of Science:

• Machine Learning
• Ensemble Methods
• Data Mining

Background:

• Random Subspace Method (RSM) builds ensemble learners using random feature subsets.
• RSM reduces learner correlation and computational load but risks using uninformative feature subsets.
• Uninformative features in subsets can negatively impact ensemble performance, especially with few informative variables.

Purpose of the Study:

• Introduce Grafted Random Subspaces (GRS) and Vanishing Random Subspaces (VRS) to address RSM's limitations.
• Enhance ensemble robustness by improving feature subset selection.
• Leverage information across trees to improve predictive accuracy.

Main Methods:

• GRS guarantees the inclusion of the most important variable from one tree into subsequent feature subsets.
• VRS guarantees the exclusion of the least important variable from subsequent feature subsets.
• Both methods build upon the Random Subspace Method by modifying feature subset selection strategies.

Main Results:

• GRS effectively recycles important features across successive trees, enhancing ensemble strength.
• VRS creates a more informative pool of candidate variables by excluding less important ones.
• These novel approaches aim to mitigate the negative impact of uninformative feature subsets in ensemble learning.

Conclusions:

• GRS and VRS offer improved ensemble performance over standard RSM.
• Intelligent feature subset management is crucial for robust machine learning ensembles.
• These methods provide effective strategies for handling datasets with a high ratio of uninformative to informative variables.