Bayes Imbalance Impact Index: A Measure of Class Imbalanced Data Set for Classification Problem

Recent studies of imbalanced data classification have shown that the imbalance ratio (IR) is not the only cause of performance loss in a classifier, as other data factors, such as small disjuncts, noise, and overlapping, can also make the problem difficult. The relationship between the IR and other data factors has been demonstrated, but to the best of our knowledge, there is no measurement of the extent to which class imbalance influences the classification performance of imbalanced data. In addition, it is also unknown which data factor serves as the main barrier for classification in a data set. In this article, we focus on the Bayes optimal classifier and examine the influence of class imbalance from a theoretical perspective. We propose an instance measure called the Individual Bayes Imbalance Impact Index (IBI³) and a data measure called the Bayes Imbalance Impact Index (BI³). IBI³ and BI³ reflect the extent of influence using only the imbalance factor, in terms of each minority class sample and the whole data set, respectively. Therefore, IBI³ can be used as an instance complexity measure of imbalance and BI³ as a criterion to demonstrate the degree to which imbalance deteriorates the classification of a data set. We can, therefore, use BI³ to access whether it is worth using imbalance recovery methods, such as sampling or cost-sensitive methods, to recover the performance loss of a classifier. The experiments show that IBI³ is highly consistent with the increase of the prediction score obtained by the imbalance recovery methods and that BI³ is highly consistent with the improvement in the F1 score obtained by the imbalance recovery methods on both synthetic and real benchmark data sets.