Progressive Training for Learning From Label Proportions.

Area of Science:

• Machine Learning
• Computer Science

Background:

• Learning from label proportions (LLP) uses group-level data to train instance-level classifiers.
• Current deep learning LLP methods use Kullback-Leibler (KL) divergence, which can lead to performance degradation due to imperfect proportion adherence.

Purpose of the Study:

• To propose a novel progressive training method (PT-LLP) for learning from label proportions.
• To address the limitations of existing methods in strictly conforming to proportion constraints.
• To improve the classification performance of deep learning-based LLP models.

Main Methods:

• PT-LLP employs a progressive training strategy, starting with KL-divergence-based methods for bag-level consistency.
• It reformulates the problem as a constrained optimization, solved using optimal transport (OT) algorithms for instance-level proportion adherence.
• Knowledge distillation within a teacher-student framework facilitates the transfer of bag-level information to the instance level.

Main Results:

• The proposed PT-LLP method achieves significant performance improvements across various datasets.
• The framework demonstrates model-agnostic capabilities, enhancing other deep LLP methods.
• The progressive approach effectively enforces proportion constraints from the bag to the instance level.

Conclusions:

• PT-LLP offers a more effective approach to learning from label proportions by addressing the limitations of existing methods.
• The method successfully integrates optimal transport and knowledge distillation for improved classification accuracy.
• This progressive training strategy represents a significant advancement in the field of machine learning for proportion-based grouped data.