Maxi-Min discriminant analysis via online learning.

Area of Science:

• Machine Learning
• Pattern Recognition
• Data Science

Background:

• Linear Discriminant Analysis (LDA) is a standard dimensionality reduction technique.
• LDA's performance degrades on multi-class data due to maximizing average, not minimal, class divergence.
• Similar classes with small divergence are often merged in LDA's reduced subspace.

Purpose of the Study:

• To introduce Maxi-Min Discriminant Analysis (MMDA), a novel dimensionality reduction method.
• To overcome LDA's limitations in handling multi-class data with similar class distributions.
• To develop a robust method for finding discriminative low-dimensional subspaces.

Main Methods:

• MMDA maximizes the minimal (worst-case) divergence among classes.
• The method is formulated as a convex optimization and large-margin learning problem.
• An efficient online learning algorithm is designed for scalability.

Main Results:

• MMDA effectively addresses the merging of similar classes inherent in LDA.
• Experimental results show MMDA outperforms five competitive approaches on various datasets.
• The method demonstrates scalability for datasets with thousands of classes.

Conclusions:

• MMDA offers a superior alternative to LDA for multi-class dimensionality reduction.
• The proposed online learning algorithm enables efficient application to large-scale data.
• MMDA provides a robust and scalable solution for complex pattern recognition tasks.