A Joint Gaussian Process Model for Active Visual Recognition with Expertise Estimation in Crowdsourcing.

Area of Science:

• Machine Learning
• Computer Vision
• Statistical Modeling

Background:

• Crowdsourcing provides large labeled datasets but suffers from noisy labels and varying labeler expertise.
• Accurate modeling of label noise and labeler quality is crucial for effective machine learning from crowds.
• Active learning strategies can optimize data labeling by selecting informative samples and high-quality labelers.

Purpose of the Study:

• To develop a noise-resilient probabilistic model for active learning from crowdsourced data.
• To explicitly model both overall label noise and individual labeler expertise.
• To enable efficient active selection of data samples and high-quality labelers.

Main Methods:

• A probabilistic model incorporating two levels of flip models to capture label noise and labeler expertise.
• Expectation propagation for approximate Bayesian inference in Gaussian process classification.
• A generalized Expectation-Maximization (EM) algorithm for estimating global label noise and individual labeler expertise.
• Prediction entropy for active sample selection and expertise-based active labeler selection.

Main Results:

• The proposed model demonstrates efficacy across four visual recognition tasks (object category, activity, gender, fine-grained classification) using real crowd-sourced data.
• An extension using the Predictive Active Set Selection Method significantly speeds up the active learning system.
• The extended model achieves higher accuracy and improved efficiency compared to baseline methods.

Conclusions:

• The developed probabilistic model effectively handles noisy crowd labels and varying labeler expertise in active learning.
• The model facilitates intelligent selection of both data samples and labelers, optimizing the learning process.
• The extended active learning system offers a more efficient and accurate approach for crowd-sourced visual recognition tasks.