Robust and Sparse Linear Discriminant Analysis via an Alternating Direction Method of Multipliers.

Area of Science:

• Machine Learning
• Pattern Recognition
• Data Science

Background:

• Traditional Linear Discriminant Analysis (LDA) is sensitive to outliers and noise.
• The L2-norm in LDA can amplify the impact of erroneous data points.
• The small sample size (SSS) problem limits LDA's effectiveness with limited data.

Purpose of the Study:

• To propose Robust Linear Discriminant Analysis (RLDA) for enhanced data analysis.
• To develop a sparse variant, RSLDA, for feature selection.
• To address the limitations of traditional LDA in handling noisy and limited datasets.

Main Methods:

• RLDA utilizes L1-norm optimization for improved robustness against outliers and noise.
• The proposed methods address nonconvex problems using the Alternating Direction Method of Multipliers (ADMM).
• RLDA and RSLDA are designed to extract an unbounded number of features, mitigating the SSS problem.

Main Results:

• RLDA demonstrates superior robustness to outliers and noise compared to standard LDA.
• RSLDA achieves sparse discriminant directions, aiding in feature interpretation.
• Both RLDA and RSLDA effectively handle the small sample size problem.
• Experiments on artificial and human face datasets validate the proposed methods.

Conclusions:

• RLDA and RSLDA offer significant improvements in robustness and feature extraction capabilities over traditional LDA.
• The L1-norm optimization in RLDA effectively reduces sensitivity to data imperfections.
• RSLDA provides a sparse solution beneficial for high-dimensional data and interpretability.