Mondrian Embeddings for Visualization of Decision Tree Ensembles

Decision trees have played a particularly important role in machine learning tasks that require explainability, interpretability, and transparency, such as medical healthcare diagnosis and disease classification. Since decision tree models can visualize decision rules for classification and regression in a tree-structured flowchart, they can help users and engineers gain an intuitive understanding of phenomena from machine learning models and judge the validity of a model's predictive diagnostics. However, in practical use of decision trees, the following two issues have become prominent, which greatly reduce their interpretability. Firstly, as the size and features/attributes of the observed data, the number of branches in the decision rules, and the number of classification classes increase, it becomes increasingly difficult for users and engineers to gain important insights from the tree structure flowchart visualization alone. Secondly, when using ensembles, a standard method for improving the predictive performance of decision trees, the flowchart structure collapses and straightforward visualization becomes ineffective. Therefore, in this paper, we propose a visualization technique that allows us to intuitively understand the findings of decision trees and their ensembles for high-dimensional and large data. Our main insight is that when two data pairs are distinguished by the decision-making process of decision trees, the early or late distinction reflects the ease with which the data is distinguished, that is, the proximity of the data. We demonstrate the advantages of our method, which can intuitively visualize not only the low-dimensional embedding of the observation data but also the behavior of the predictor in the same space, using five biology datasets.