Training Complex Models with Multi-Task Weak Supervision.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Data Science

Background:

• Collecting large hand-labeled training datasets is a major bottleneck for complex machine learning models.
• Weak supervision sources offer cheaper, noisier labels but have diverse, unknown accuracies and may be correlated or applied at different granularities.
• Existing methods often model weak supervision sources separately, failing to leverage potential task relationships.

Purpose of the Study:

• To propose a novel framework for integrating and modeling diverse weak supervision sources within a multi-task weak supervision setting.
• To recover the accuracies of multi-task weak supervision sources without labeled data by solving a matrix completion problem.
• To improve the quality of supervision for training end machine learning models.

Main Methods:

• Framing weak supervision sources as labeling related sub-tasks within a multi-task weak supervision setting.
• Employing a matrix completion-style approach to estimate source accuracies based on their dependency structure, using unlabeled data.
• Theoretically analyzing the generalization error and its scaling with unlabeled data, task structure, and dependency structure.

Main Results:

• The proposed multi-task weak supervision framework recovers source accuracies without labeled data.
• Theoretical analysis shows improved generalization error with more unlabeled data.
• Empirical results on three fine-grained classification tasks demonstrate significant accuracy gains: 20.2 points over traditional supervised learning, 6.8 points over majority vote, and 4.1 points over separate task modeling.

Conclusions:

• The multi-task weak supervision approach effectively integrates diverse weak label sources, leading to higher-quality supervision.
• This method significantly enhances model accuracy compared to existing approaches, especially when labeled data is scarce.
• The framework provides a robust solution for leveraging weak supervision in complex machine learning tasks.