Learning regularized LDA by clustering.

Area of Science:

• Computer Science
• Machine Learning
• Pattern Recognition

Background:

• Supervised dimensionality reduction techniques like linear discriminant analysis (LDA) suffer from overfitting with limited training samples per class.
• This overfitting stems from inaccurate estimations of between- and within-class scatter matrices due to small sample sizes.

Purpose of the Study:

• To propose a novel regularization method for LDA that mitigates overfitting without requiring more training data.
• To enhance the robustness of LDA by leveraging the inherent structure of the training data.

Main Methods:

• Regularizing the between- and within-class scatter matrices using simultaneously computed between- and within-cluster scatter matrices.
• Computing cluster-based scatter matrices from unsupervised clustered data.
• Weighting the contributions of cluster scatter matrices inversely proportional to the number of training samples per class.

Main Results:

• The proposed method effectively regularizes scatter matrices, improving LDA performance on datasets with few samples per class.
• Experimental validation on AR and Feret face databases demonstrated significant improvements in classification accuracy.
• The benefits of the method are more pronounced as the number of training samples per class decreases.

Conclusions:

• The proposed cluster-regularized LDA offers a robust solution to the overfitting problem in supervised dimensionality reduction.
• This technique is particularly advantageous for applications with limited labeled data, such as face recognition.
• The method enhances class discrimination and data representation by utilizing unsupervised clustering structures.