Average Top-k Aggregate Loss for Supervised Learning.

Area of Science:

• Machine Learning
• Supervised Learning
• Optimization

Background:

• Standard aggregate losses like average loss and maximum loss have limitations in adapting to varied data distributions.
• Existing methods may not offer sufficient flexibility for optimizing supervised learning models across diverse datasets.

Purpose of the Study:

• Introduce the average top-k (ATk) loss as a novel aggregate loss function for supervised learning.
• Demonstrate the ATk loss's ability to generalize existing aggregate losses and adapt to different data distributions.
• Investigate the theoretical properties and practical applicability of the ATk loss in classification and regression tasks.

Main Methods:

• Defined the ATk loss as the average of the k largest individual losses in a training set.
• Analyzed the ATk loss's convexity and its compatibility with various individual loss functions.
• Studied classification calibration and derived error bounds for ATk-SVM models.
• Conducted experiments on synthetic and real-world datasets for binary/multi-class classification and regression.

Main Results:

• The ATk loss serves as a natural generalization of average and maximum losses.
• ATk loss offers enhanced adaptability to different data distributions through the parameter k.
• The ATk loss maintains convexity and computational efficiency.
• Empirical results validate the effectiveness of ATk loss in diverse supervised learning scenarios.

Conclusions:

• The ATk loss provides a flexible and robust alternative to existing aggregate loss functions.
• Minimum average top-k learning is applicable to a wide range of supervised learning problems.
• The ATk loss framework offers improved performance and adaptability in machine learning models.