Prepare for the Worst, Hope for the Best: Active Robust Learning On Distributions

Area of Science:

• Machine Learning
• Statistical Learning Theory
• Distributional Data Analysis

Background:

• Advanced learning systems now handle complex data, including distributions as individual examples.
• Learning on distributions is challenging due to indirect access via samples, leading to inexact estimates.
• Robust learning principles are crucial for handling these inexact data representations.

Purpose of the Study:

• To propose an active robust learning framework for distributional data.
• To develop a method that minimizes expected risk by selecting informative samples.
• To address the challenges of inexact distributional estimates in active learning.

Main Methods:

• Derivation of an upper bound on classifier risk in active learning stages.
• Proposal of Probabilistic Minimax Active Learning (PMAL), a Bayesian multiclass active learning strategy.
• Development of an efficient approximation for an intractable objective function using convex optimization.
• Integration of kernel embedding of distributions via a Bayesian method for robust learning.

Main Results:

• The proposed PMAL method provably selects samples that minimize expected risk.
• An efficient approximation with a known error bound is presented for computational tractability.
• A novel active robust learning on distributions method is introduced, leveraging kernel embeddings.
• Experimental validation on synthetic and real-world datasets demonstrates method effectiveness.

Conclusions:

• The developed active robust learning on distributions method is effective for analyzing complex distributional datasets.
• PMAL offers a practical and theoretically grounded approach to active learning with inexact distributional data.
• The study advances robust learning techniques for higher-level data structures.