Discrete Box-Constrained Minimax Classifier for Uncertain and Imbalanced Class Proportions

Area of Science:

• Machine Learning
• Statistical Classification

Background:

• Supervised classifiers struggle with imbalanced datasets, uncertain class proportions, feature dependencies, mixed data types, and arbitrary loss functions.
• Bayes classifiers are sensitive to prior probability shifts between training and testing data.
• Equalizer decision rules aim to equalize class-conditional risks, a concept related to minimax classifiers that maximize Bayes risk.

Purpose of the Study:

• To develop a novel box-constrained minimax classifier robust to imbalanced datasets and uncertain priors.
• To address challenges including feature dependencies, mixed data types (numeric and categorical), and arbitrary loss functions.
• To create a classifier that controls risk maximization by incorporating constraints on priors.

Main Methods:

• Development of a box-constrained minimax classifier incorporating prior constraints.
• Analysis of empirical Bayes risk for discrete inputs, identifying it as a concave, non-differentiable, multivariate piecewise affine function.
• Derivation and convergence analysis of a projected subgradient algorithm for risk maximization over a box-constrained simplex, demonstrating scalability for large numbers of classes.

Main Results:

• The projected subgradient algorithm guarantees convergence with bounded speed.
• The optimization algorithm is scalable, particularly for scenarios with a large number of classes.
• The proposed classifier demonstrates robustness across diverse databases when combined with a clustering algorithm for mixed attributes.

Conclusions:

• The novel box-constrained minimax classifier effectively equalizes class-conditional risks.
• The classifier exhibits robustness and controlled risk maximization, even with uncertain prior probabilities.
• The integrated approach using the classifier and a clustering algorithm handles mixed attributes efficiently.