Deep Neural Network Initialization With Decision Trees.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Deep feedforward neural networks (DNNs) are powerful predictive models but can be computationally expensive to train.
• Decision trees offer interpretable models but may lack the scalability of DNNs for complex datasets.

Purpose of the Study:

• To present a novel, automated method for constructing and initializing DNNs using decision trees.
• To introduce deep jointly informed neural networks (DJINN) as an efficient and accurate modeling approach.

Main Methods:

• Mapping decision trees trained on data to initialized neural networks.
• Utilizing tree structures to determine neural network architectures.
• Employing tree-informed initialization as a warm-start for DNN training.

Main Results:

• DJINN models demonstrate high predictive performance on diverse regression and classification tasks.
• Achieved comparable performance to Bayesian hyperparameter optimization.
• Lower computational cost compared to Bayesian optimization.

Conclusions:

• DJINN effectively combines the interpretability of decision trees with the power of DNNs.
• This approach offers an attractive alternative for training predictive models on complex datasets.
• DJINN provides efficient training and high accuracy for machine learning applications.