Snorkel: Rapid Training Data Creation with Weak Supervision.

Area of Science:

• Machine Learning
• Data Science
• Artificial Intelligence

Background:

• Labeling training data is a major bottleneck in machine learning deployment.
• Current methods often require extensive manual data annotation, which is time-consuming and costly.

Purpose of the Study:

• Introduce Snorkel, a novel system for training state-of-the-art models without hand-labeled data.
• Enable users to leverage heuristics through labeling functions for data annotation.

Main Methods:

• Implement a data programming paradigm for denoising outputs from labeling functions without ground truth.
• Develop a flexible interface for writing labeling functions based on user collaboration.
• Propose an optimizer to automate modeling tradeoff decisions.

Main Results:

• Subject matter experts built models 2.8x faster and improved performance by 45.5% compared to hand labeling.
• Snorkel achieved 132% average improvement over prior heuristic methods on real-world datasets.
• Performance was within 3.60% of large hand-curated training sets.

Conclusions:

• Snorkel offers a powerful alternative to traditional data labeling, accelerating machine learning development.
• The data programming approach effectively denoises heuristic-generated labels, leading to significant performance gains.
• Snorkel demonstrates practical utility in collaborations with government agencies and on diverse datasets.