Deep forest.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Current deep learning models predominantly rely on neural networks trained via backpropagation.
• Traditional machine learning techniques like decision trees and boosting machines offer alternative approaches.
• The success of deep neural networks is often attributed to layer-by-layer processing, in-model feature transformation, and model complexity.

Purpose of the Study:

• To explore deep models built upon non-differentiable modules, specifically decision trees.
• To propose a novel deep learning architecture inspired by characteristics of deep neural networks.
• To offer an alternative to gradient-based deep learning methods.

Main Methods:

• Developed a decision-tree ensemble approach termed 'deep forest'.
• Ensured the deep forest model possesses layer-by-layer processing, in-model feature transformation, and data-dependent complexity.
• Minimized hyper-parameter dependence compared to deep neural networks.

Main Results:

• The deep forest model demonstrated robust performance across diverse datasets and domains.
• Excellent results were achieved consistently, even with default hyper-parameter settings.
• The approach proved effective without relying on gradient-based adjustments.

Conclusions:

• Deep learning can be achieved using non-differentiable modules, challenging the necessity of backpropagation.
• Deep forests offer a viable, robust, and less hyper-parameter-sensitive alternative to deep neural networks.
• This research opens new avenues for deep learning architectures beyond traditional neural networks.