Does Tail Label Help for Large-Scale Multi-Label Learning?

Area of Science:

• Machine Learning
• Data Science
• Artificial Intelligence

Background:

• Large-scale multi-label learning (LMLL) involves annotating unseen data with relevant labels from a vast set of candidates.
• Labels in LMLL often follow a long-tail distribution, with numerous infrequent 'tail' labels.
• Previous research assumed tail labels improve performance, but their specific impact remained unquantified.

Purpose of the Study:

• To quantify the impact of tail labels on LMLL performance metrics.
• To develop efficient LMLL methods by identifying and leveraging less influential labels.
• To reduce computational complexity, prediction time, and model size in LMLL.

Main Methods:

• Analyzed the impact of label frequency and weight on LMLL evaluation metrics like Propensity Score Precision (PSP)@k and Propensity Score nDCG (PSnDCG)@k.
• Proposed low-complexity LMLL methods that strategically restrain less performance-influential labels.
• Developed a technique to preserve dominant model parameters for these less influential labels to mitigate performance loss.

Main Results:

• Demonstrated that label impact on PSP@k and PSnDCG@k is proportional to the product of label weights and frequencies.
• Showcased that tail labels, especially with equal weights, have significantly less impact due to lower frequencies.
• Empirically validated that the proposed methods substantially reduce prediction time and model size without significant performance degradation.

Conclusions:

• The impact of labels in LMLL is quantifiable and directly related to their frequency and weight.
• Low-complexity LMLL methods can be developed by focusing on high-impact labels, leading to efficiency gains.
• The proposed approach effectively balances model efficiency with predictive performance in large-scale multi-label learning scenarios.