Crowd labeling latent Dirichlet allocation.

Area of Science:

• Machine Learning
• Data Science
• Computational Statistics

Background:

• Large unlabeled datasets are prevalent, but manual labeling is costly and time-consuming.
• Crowd labeling utilizes crowdsourcing for data annotation, though worker accuracy varies.
• Existing algorithms for crowd labeling have limitations in handling complex labeling tasks.

Purpose of the Study:

• To introduce crowd labeling latent Dirichlet allocation (CL-LDA) as a generalized solution for crowd labeling problems.
• To demonstrate CL-LDA's effectiveness compared to existing methods on diverse datasets.
• To incorporate prior knowledge of worker abilities into the labeling model.

Main Methods:

• Developed crowd labeling latent Dirichlet allocation (CL-LDA), a novel generalization of latent Dirichlet allocation.
• Treated labels as compositional, moving beyond discrete class assignments.
• Integrated worker expertise using a structured Bayesian framework.
• Applied CL-LDA to the EEG independent component labeling dataset.

Main Results:

• CL-LDA performed comparably to, and sometimes better than, existing methods on simulated and real-world datasets.
• The compositional label approach proved effective for complex labeling scenarios.
• The model successfully incorporated prior knowledge of worker abilities.
• Demonstrated the utility of CL-LDA and its generalizations on the EEG dataset.

Conclusions:

• CL-LDA provides a robust and flexible framework for crowd labeling tasks.
• The method's ability to handle compositional labels and worker expertise enhances data annotation.
• CL-LDA shows promise for generating reliable labels for downstream classification tasks, particularly in domains like EEG analysis.