Random Shapley Forests: Cooperative Game-Based Random Forests With Consistency.

Area of Science:

• Machine Learning
• Computational Statistics

Background:

• The original random forests (RFs) algorithm is widely applied but lacks theoretical depth.
• A gap exists between the practical applications and theoretical understanding of RFs.

Purpose of the Study:

• To propose a novel random forests algorithm, random Shapley forests (RSFs), to enhance the theoretical foundation of RFs.
• To leverage Shapley values for a more robust evaluation of feature importance within the RF framework.

Main Methods:

• Developed the random Shapley forests (RSFs) algorithm, integrating Shapley value calculations at each tree node.
• Evaluated feature importance by assessing dependencies within feature coalitions.
• Proved the consistency of the RSFs algorithm, drawing inspiration from existing consistency theories.

Main Results:

• RSFs demonstrated superior or comparable performance against original RFs, consistent RFs, and support vector machines.
• Experiments were conducted on diverse datasets, including eight UCI benchmarks and four real-world datasets.
• The proposed RSFs algorithm shows effectiveness in classification and regression tasks.

Conclusions:

• RSFs offer a theoretically grounded advancement over traditional RFs.
• The integration of Shapley values provides a fair and effective method for feature importance assessment.
• RSFs represent a promising development in ensemble learning methods.