Discriminant Feature Extraction by Generalized Difference Subspace.

Area of Science:

• Computer Science
• Machine Learning
• Pattern Recognition

Background:

• Generalized Difference Subspace (GDS) projection effectively extracts features for subspace-based classifiers by quasi-orthogonalizing class subspaces.
• Fisher Discriminant Analysis (FDA) is a powerful technique for discriminant feature extraction, but suffers from the small sample size problem.

Purpose of the Study:

• To theoretically and experimentally reveal the discriminant capacity of GDS projection.
• To establish a connection between GDS projection and FDA.
• To introduce and validate extensions of GDS projection for enhanced performance, particularly in small-sample scenarios.

Main Methods:

• Introduction of Geometrical Fisher Discriminant Analysis (gFDA) based on a simplified Fisher criterion and a heuristic assumption relating class mean vectors to principal component vectors.
• Theoretical proof establishing the equivalence between gFDA and GDS projection with a correction term.
• Development and application of nonlinear extensions using kernel tricks and integration with Convolutional Neural Network (CNN) features.

Main Results:

• GDS projection demonstrates discriminant capacity analogous to FDA through the mechanism of gFDA.
• gFDA provides a stable alternative to FDA, effectively addressing the small sample size problem.
• The proposed extensions (kernelized GDS and GDS with CNN features) significantly improve performance in image recognition tasks with limited data.

Conclusions:

• GDS projection possesses inherent discriminant capabilities, effectively bridging the gap between subspace methods and FDA.
• The developed gFDA offers a robust approach for discriminant analysis, especially when sample sizes are small.
• The nonlinear and CNN-based extensions of GDS projection provide powerful tools for challenging image recognition problems with limited training data.