Active Learning of Multi-Class Classifiers with Auxiliary Probabilistic Information

Our ability to learn accurate classification models from data is often limited by the number of available data instances. This limitation is of particular concern when data instances need to be labeled by humans and when the labeling process carries a significant cost. Recent years witnessed increased research interest in developing methods capable of learning models from a smaller number of examples. One such direction is active learning. Another, more recent direction showing a great promise utilizes auxiliary probabilistic information in addition to class labels. However, this direction has been applied and tested only in binary classification settings. In this work we first develop a multi-class variant of the auxiliary probabilistic approach, and after that embed it within an active learning framework, effectively combining two strategies for reducing the dependency of multi-class classification learning on the number of labeled examples. We demonstrate the effectiveness of our new approach on both simulated and real-world datasets.