Chained Deep Learning Using Generalized Cross-Entropy for Multiple Annotators Classification.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Data Science

Background:

• Supervised learning relies on accurate data labeling, often by experts.
• Crowdsourcing offers a cost-effective solution for large datasets but faces challenges with annotator variability and noise.
• Traditional methods struggle with non-homogeneous annotator behavior and labeler inter-dependencies.

Purpose of the Study:

• To propose a novel framework, Generalized Cross-Entropy-based framework using Chained Deep Learning (GCECDL), to address limitations in crowdsourced data labeling.
• To model non-stationary annotator patterns and preserve inter-expert dependencies for robust classification.
• To enhance predictive performance and provide interpretability of annotator trustworthiness.

Main Methods:

• Developed a Generalized Cross-Entropy-based framework using Chained Deep Learning (GCECDL).
• Coded individual annotator non-stationary patterns relative to the input space.
• Preserved inter-dependencies among experts via a chained deep learning architecture.
• Integrated a noise-robust loss function and network self-regularization by estimating labeler reliability.

Main Results:

• GCECDL demonstrated robust predictive properties on multiple-annotator classification tasks.
• Outperformed state-of-the-art algorithms in handling noisy labels and varying annotator expertise.
• Achieved network self-regularization by estimating labeler reliability within the chained approach.
• Visual inspection and relevance analysis confirmed the method's non-stationary coding capabilities.

Conclusions:

• GCECDL effectively weights reliable labelers based on input samples, enhancing discrimination performance.
• The framework offers preserved interpretability regarding individual annotator trustworthiness.
• This approach combines deep learning power with noise robustness for improved crowdsourced data labeling.