Direct nonparametric predictive inference classification trees

Classification is the task of assigning a new instance to one of a set of predefined categories based on the attributes of the instance. A classification tree is one of the most commonly used techniques in the area of classification. In this paper, we introduce a novel classification tree algorithm which we call Direct Nonparametric Predictive Inference (D-NPI) classification algorithm. The D-NPI algorithm is completely based on the Nonparametric Predictive Inference (NPI) approach, and it does not use any other assumptions. NPI is a statistical methodology which learns from data in the absence of prior knowledge and uses only few modelling assumptions, enabled by the use of lower and upper probabilities to quantify uncertainty. Due to the predictive nature of NPI, it is well suited for classification, as the nature of classification is explicitly predictive as well. The D-NPI algorithm uses a new split criterion called Correct Indication (CI). CI reflects how informative attribute variables are, hence if the attribute variable is very informative, it gives high NPI lower and upper probabilities for CI. In addition, CI reports the strength of the evidence that the attribute variables will indicate regarding the possible class state for future instances, based on the data. To demonstrate its real-world applicability, the D-NPI algorithm is tested on benchmark data sets from various domains obtained from the UCI machine learning repository. The performance of the D-NPI classification algorithm is tested against several other classification algorithms using classification accuracy, in-sample accuracy and tree size. The experimental results indicate that the D-NPI classification algorithm performs well and tends to slightly outperform the other classification algorithms.