Efficient Learning of Classification Models from Soft-label Information by Binning and Ranking

Construction of classification models from data in practice often requires additional human effort to annotate (label) observed data instances. However, this annotation effort may often be too costly and only a limited number of data instances may be feasibly labeled. The challenge is to find methods that let us reduce the number of the labeled instances but at the same time preserve the quality of the learned models. In this paper we study the idea of learning classification from soft label information in which each instance is associated with a soft-label further refining its class label. One caveat of applying this idea is that soft-labels based on human assessment are often noisy. To address this problem, we develop and test a new classification model learning algorithm that relies on soft-label binning to limit the effect of soft-label noise. We show this approach is able to learn classification models more rapidly and with a smaller number of labeled instances than (1) existing soft label learning methods, as well as, (2) methods that learn from class-label information.