Collaborative Active Visual Recognition from Crowds: A Distributed Ensemble Approach.

Area of Science:

• Machine Learning
• Computer Vision
• Human-Computer Interaction

Background:

• Active learning enhances model training through user interaction.
• Existing research primarily focuses on single human oracle active learning.
• Collaborative active learning with multiple oracles, especially with noisy inputs, is underexplored.

Purpose of the Study:

• To develop a collaborative computational model for active learning with multiple human oracles.
• To address the challenge of varying noise levels in inputs from multiple oracles.
• To create a system robust to label noise and capable of assessing labeler quality.

Main Methods:

• Developed an ensemble kernel machine robust to label noise.
• Integrated a principled label quality measure for online detection of irresponsible labelers.
• Modeled correlations among labelers by sharing data, avoiding independent active learning processes.

Main Results:

• The proposed model demonstrates robustness against noisy labels from multiple oracles.
• Successfully implemented an online mechanism to detect unreliable labelers.
• Experiments with simulated and real crowd-sourced data validate the model's efficacy.

Conclusions:

• The collaborative active learning model effectively trains visual recognition systems with multiple noisy oracles.
• The approach provides a robust solution for handling label noise and assessing labeler reliability in crowd-sourced environments.
• This work advances collaborative active learning by considering inter-labeler correlations and noise.