Learning From Crowds With Multiple Feature Dynamic Fusion-Based Annotation Generation.

Area of Science:

• Machine Learning
• Data Annotation
• Crowdsourcing

Background:

• Accurate data annotation for machine learning is costly and time-consuming.
• Crowdsourcing is a popular method for data collection and annotation, but introduces label noise.
• Existing methods for handling label noise struggle with sparse crowdsourced data, limiting worker capability modeling.

Purpose of the Study:

• To propose a novel network, Multi-Feature Dynamic Fusion Annotation Generation Network (MFDFAGen-Net), to address label noise in crowdsourced datasets.
• To dynamically rectify the learning process by leveraging correlations between noise transition and instance dependency.
• To enhance the modeling of worker capabilities and instance reliability.

Main Methods:

• Developed MFDFAGen-Net, a network that dynamically integrates worker and instance features.
• Implemented a mechanism where worker-capability and instance-rectification confusion matrices optimize each other.
• Fused two confusion matrices with multi-feature representations for improved expression of worker capabilities.

Main Results:

• Theoretical analysis shows MFDFAGen-Net effectively learns instance and worker reliability and bias.
• Extensive experiments on synthetic and real-world datasets demonstrate significant performance improvements.
• MFDFAGen-Net outperforms several state-of-the-art methods in handling crowdsourced label noise.

Conclusions:

• MFDFAGen-Net offers a robust solution for mitigating label noise in machine learning datasets derived from crowdsourcing.
• The proposed model provides more fine-grained knowledge of worker and instance characteristics.
• This approach enhances the overall performance and reliability of machine learning models trained on noisy crowdsourced data.