Correlated Chained Gaussian Processes for Datasets With Multiple Annotators.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Data Science

Background:

• Supervised learning typically relies on expert-labeled data (ground truth).
• Real-world scenarios often involve data labeled by multiple annotators with varying, unknown expertise levels.
• Existing Learning from Crowds (LFC) methods often assume annotator performance is independent of the input space and that annotators are independent.

Purpose of the Study:

• To develop an advanced LFC approach that addresses limitations of current methods.
• To model individual annotator performance as a function of the input space.
• To leverage correlations among multiple annotators for improved learning.

Main Methods:

• Proposed the Correlated Chained Gaussian Processes from Multiple Annotators (CCGPMA) model.
• CCGPMA models annotator performance dynamically based on input features.
• The approach explicitly models inter-annotator dependencies.

Main Results:

• CCGPMA demonstrated superior modeling of labeler behavior compared to existing LFC methods.
• Experimental results on classification and regression tasks showed consistent performance improvements.
• The proposed method effectively captures complex annotator characteristics.

Conclusions:

• CCGPMA offers a more robust framework for Learning from Crowds.
• The model's ability to account for input-dependent performance and inter-annotator correlations enhances accuracy.
• This approach advances the field of machine learning with noisy, crowd-sourced labels.