Active Learning of Classification Models with Likert-Scale Feedback.

Area of Science:

• Machine Learning
• Data Science
• Computer Science

Background:

• Human annotation of classification data is labor-intensive and time-consuming.
• Efficient annotation is crucial for practical classification model development and diverse applications.
• Existing methods often rely solely on active learning or limited feedback scales.

Purpose of the Study:

• To develop a novel active learning framework to significantly reduce annotation effort.
• To integrate human Likert-scale feedback with active learning for enhanced efficiency.
• To accelerate the training of classification models while minimizing the need for labeled data.

Main Methods:

• A new active learning framework combining label uncertainty from Likert-scale feedback.
• Utilizing active learning to select examples with the greatest expected change for annotation.
• Employing a Bayesian approach for expectation calculation and an incremental SVM solver for efficiency.

Main Results:

• The proposed framework demonstrates faster learning compared to methods using only Likert-scale feedback or active learning.
• Achieves comparable or superior model performance with a substantially reduced number of labeled instances.
• Effectively leverages human judgment and active learning to optimize the annotation process.

Conclusions:

• The integration of active learning and Likert-scale feedback offers a powerful strategy for efficient data annotation.
• This approach significantly reduces the cost and time associated with building practical classification models.
• Enables broader application of classification models by lowering the barrier to data labeling.