Data Subset Selection With Imperfect Multiple Labels.

Area of Science:

• Machine Learning
• Data Science
• Computer Science

Background:

• Real-world applications often use redundant, imperfect labels for data instances.
• Acquiring numerous low-cost, less-than-expert labels is common for ground truth estimation.
• Noisy labels degrade supervised learning performance, and data preparation can be costly.

Purpose of the Study:

• To introduce a novel quality control mechanism for data subset selection.
• To address the challenge of selecting optimal subsets from weakly labeled data.
• To improve the efficiency and effectiveness of supervised learning with imperfect labels.

Main Methods:

• Developed a quality control mechanism to estimate labeling quality per instance.
• Applied the probably approximately correct (PAC) model for data subset selection.
• Proposed algorithms for selecting k instances with high expected labeling quality.

Main Results:

• Demonstrated substantial benefits of using a reliable subset over all imperfectly labeled data.
• Showcased expected labeling quality as an effective indicator for allocating labeling effort.
• Validated the algorithms' effectiveness in determining optimal data subsets and acquisition needs.

Conclusions:

• The proposed quality control mechanism and selection algorithms efficiently identify valuable data subsets.
• Expected labeling quality guides targeted data acquisition, optimizing resource allocation.
• The approach significantly enhances supervised learning performance compared to using all available noisy data.