Crowdsourced Label Aggregation Using Bilayer Collaborative Clustering.

Area of Science:

• Machine Learning
• Data Science
• Crowdsourcing

Background:

• Online crowdsourcing platforms offer cost-effective label acquisition from nonexpert workers.
• Aggregating multiple noisy labels is standard practice to improve data quality.
• Traditional methods struggle to enhance label accuracy significantly when initial labeling quality is low.

Purpose of the Study:

• To propose a novel bilayer collaborative clustering (BLCC) method for improved label aggregation in crowdsourcing.
• To address the limitations of traditional label aggregation techniques in low-quality labeling scenarios.
• To enhance the accuracy of integrated labels derived from imperfect crowdsourced data.

Main Methods:

• The proposed BLCC method utilizes a two-layer clustering approach: conceptual and physical layers.
• Conceptual-level features are generated from multiple noisy labels for initial clustering and label inference.
• Physical-level clustering refines label estimations, with both layers iteratively guiding each other to track uncertainties and remedy errors.

Main Results:

• Experimental results on 12 real-world crowdsourcing datasets demonstrate superior performance compared to state-of-the-art methods.
• The BLCC method shows significant improvements in the accuracy of integrated labels.
• The dual-layer clustering effectively manages and reduces label uncertainties.

Conclusions:

• The BLCC method offers a robust solution for label aggregation in crowdsourcing, particularly in challenging low-quality labeling situations.
• The collaborative nature of the dual-layer clustering enhances the reliability and accuracy of the final aggregated labels.
• BLCC provides a significant advancement over existing label aggregation techniques, paving the way for more dependable crowdsourced datasets.