Maxi-min margin machine: learning large margin classifiers locally and globally

Area of Science:

• Machine Learning
• Pattern Recognition
• Computational Statistics

Background:

• Existing large margin classifiers like Support Vector Machines (SVM) and Minimax Probability Machines (MPM) focus on either local or global data information.
• A unified approach considering both local and global data perspectives is lacking.

Purpose of the Study:

• To propose a novel large margin classifier, the maxi-min margin machine (M4), that integrates both local and global data information.
• To demonstrate the theoretical relationships between M4, SVM, and MPM.
• To extend M4 for nonlinear classification using Mercer kernels.

Main Methods:

• Definition of the M4 model and its geometrical interpretation.
• Theoretical analysis of M4, including its relationship to SVM and MPM.
• Development of a sequential conic programming method for optimization.
• Application of Mercer kernels for nonlinear classification.

Main Results:

• M4 learns decision boundaries using both local and global data information, unlike SVM (local) and MPM (global).
• SVM is shown to be a special case of M4 under certain conditions.
• MPM is identified as a relaxation of M4.
• M4 demonstrates advantages over SVM and MPM in evaluations on synthetic and real-world datasets.

Conclusions:

• The maxi-min margin machine (M4) offers a superior approach to large margin classification by incorporating both local and global data perspectives.
• M4 provides a unified framework that enhances understanding and extends existing methods like SVM, MPM, and Linear Discriminant Analysis.
• Empirical results validate the effectiveness and advantages of the M4 model.