Reliable Programmatic Weak Supervision With Confidence Intervals for Label Probabilities.

Area of Science:

• Machine Learning
• Data Science
• Artificial Intelligence

Background:

• Accurate dataset labeling is expensive and time-consuming.
• Programmatic weak supervision uses multiple labeling functions (LFs) for probabilistic predictions.
• Existing methods lack reliability assessments for probabilistic labels.

Purpose of the Study:

• To develop a programmatic weak supervision methodology offering confidence intervals for label probabilities.
• To enhance the reliability of predictions from weak labeling functions.
• To address the challenge of assorted LF types and unknown interdependences.

Main Methods:

• Utilizing uncertainty sets of distributions to model LF information.
• Encapsulating information from LFs with unrestricted behavior and typology.
• Developing a framework for programmatic weak supervision with confidence estimation.

Main Results:

• Demonstrated improved prediction reliability compared to state-of-the-art methods.
• Showcased the practicality and utility of generated confidence intervals.
• Validated through experiments on multiple benchmark datasets.

Conclusions:

• The proposed methodology enhances programmatic weak supervision by providing reliable predictions and confidence intervals.
• The approach effectively handles diverse and interdependent weak labeling functions.
• This work offers a significant advancement in creating trustworthy probabilistic labels for datasets.