TNT: An Interpretable Tree-Network-Tree Learning Framework using Knowledge Distillation.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Explainable AI

Background:

• Deep Neural Networks (DNNs) offer high performance but lack decision process interpretability.
• Interpretability is critical in sensitive fields like medical and financial data analysis.
• Existing DNNs present an ambiguous decision process for individual test cases.

Purpose of the Study:

• To propose a Tree-Network-Tree (TNT) learning framework for explainable decision-making.
• To enhance the interpretability of DNNs through knowledge transfer with tree models.
• To develop a method that combines the strengths of tree-based models and DNNs for interpretable predictions.

Main Methods:

• Proposed a novel James-Stein Decision Tree (JSDT) for improved knowledge representation, especially with low-quality data.
• Utilized DNNs as a teacher model to embed knowledge for subsequent tree models.
• Introduced a distillable Gradient Boosted Decision Tree (dGBDT) to learn interpretable trees from DNN soft labels.

Main Results:

• The TNT framework demonstrated effectiveness across various machine learning tasks.
• The JSDT improved knowledge embedding for DNNs.
• The dGBDT achieved comparable predictions to DNNs while providing interpretability.

Conclusions:

• The TNT learning framework successfully integrates DNNs and decision trees for explainable AI.
• The proposed method offers a viable solution for scenarios requiring interpretable predictions.
• The TNT framework enhances decision-making transparency in machine learning applications.