Large-Scale Nonlinear AUC Maximization via Triply Stochastic Gradients

Area of Science:

• Machine Learning
• Data Science
• Artificial Intelligence

Background:

• Improving Area Under the Curve (AUC) for imbalanced data is a key machine learning challenge.
• Existing AUC maximization methods often assume linear models, limiting effectiveness on nonlinear problems.
• Scaling nonlinear AUC maximization remains an open research question.

Purpose of the Study:

• To propose a novel, large-scale nonlinear AUC maximization method.
• To address the limitations of existing methods in handling nonlinear separable problems.
• To provide an efficient and scalable solution for nonlinear AUC maximization.

Main Methods:

• The proposed method, TSAM (Triply Stochastic AUC Maximization), utilizes random Fourier features to approximate kernel functions.
• It employs triply stochastic gradients with respect to pairwise loss and random features for iterative updates.
• Convergence analysis proves an optimal solution rate of O(1/t) after t iterations.

Main Results:

• TSAM demonstrates scalability for large-scale nonlinear AUC maximization.
• Experimental results show significant reductions in computational time compared to batch learning algorithms.
• The method achieves comparable generalization performance to existing approaches.

Conclusions:

• TSAM offers an effective and scalable solution for nonlinear AUC maximization on imbalanced datasets.
• The approach overcomes the limitations of linear assumptions in traditional AUC optimization.
• TSAM presents a promising advancement in handling complex, nonlinear data patterns for improved model performance.