Efficient Low-Rank Semidefinite Programming With Robust Loss Functions.

Area of Science:

• Machine Learning
• Optimization
• Robust Statistics

Background:

• Real-world machine learning requires algorithms resilient to data outliers and corruptions.
• Traditional low-rank semi-definite programming (SDP) formulations often use square loss, which is highly sensitive to such anomalies.
• Improving robustness in SDP is crucial for reliable practical applications.

Purpose of the Study:

• To enhance the robustness of machine learning algorithms formulated as low-rank SDP problems.
• To address the challenges posed by non-convex and non-smooth optimization objectives arising from robust loss functions.
• To develop an efficient and theoretically guaranteed optimization algorithm.

Main Methods:

• Proposed replacing the sensitive square loss with robust noise models, specifically the l1-loss and other non-convex losses.
• Designed an efficient algorithm utilizing majorization-minimization techniques.
• Employed the alternating direction method of multipliers (ADMM) to construct an efficient optimization surrogate for the non-convex problem.
• Incorporated convergence monitoring for ADMM to ensure theoretical guarantees.

Main Results:

• The developed algorithm is empirically efficient and theoretically guaranteed to converge to a critical point.
• Extensive experiments on four machine learning applications demonstrated the algorithm's speed.
• The proposed method achieved superior performance compared to existing state-of-the-art approaches on both synthetic and real-world datasets.
• The algorithm effectively handles non-convex and non-smooth optimization landscapes.

Conclusions:

• The proposed majorization-minimization algorithm effectively improves the robustness of low-rank SDP learning algorithms.
• The integration of robust loss functions and ADMM provides a computationally efficient and reliable solution for outlier-prone data.
• This approach offers a significant advancement over traditional methods, showing enhanced performance and speed in practical machine learning tasks.