Robust recursive absolute value inequalities discriminant analysis with sparseness.

Area of Science:

• Machine Learning
• Computer Vision
• Data Science

Background:

• Supervised dimensionality reduction aims to find discriminative features.
• Conventional Linear Discriminant Analysis (LDA) is sensitive to outliers and noise.
• The SSS problem can limit the effectiveness of LDA.

Purpose of the Study:

• To propose a novel criterion for supervised dimensionality reduction called Absolute Value Inequalities Discriminant Analysis (AVIDA).
• To enhance robustness against outliers and noise compared to traditional LDA.
• To achieve sparse discriminant vectors for improved interpretability and efficiency.

Main Methods:

• Reformulated the generalized eigenvalue problem in LDA into a Support Vector Machine (SVM)-type concave-convex problem using absolute value inequalities loss.
• Incorporated an L1-norm regularization term to promote sparsity in discriminant vectors.
• Employed a successive linear algorithm, solving a series of linear programs to address the optimization problem.

Main Results:

• AVIDA demonstrated increased robustness to outliers and noise.
• The method effectively avoided the SSS problem inherent in some LDA formulations.
• Sparse discriminant vectors were successfully obtained.
• Experimental results on artificial and benchmark image datasets validated AVIDA's superiority.

Conclusions:

• AVIDA offers a robust and sparse alternative for supervised dimensionality reduction.
• The proposed method enhances feature extraction by mitigating the impact of noisy data.
• AVIDA shows significant potential for applications in image analysis and other data-driven fields.