Feature extraction with deep neural networks by a generalized discriminant analysis

Area of Science:

• Machine Learning
• Computer Vision
• Pattern Recognition

Background:

• Classical Linear Discriminant Analysis (LDA) assumes linear separability, limiting its effectiveness with complex, arbitrarily distributed data.
• Linear transformations are insufficient for optimal discriminative feature extraction from raw, high-dimensional measurements.

Purpose of the Study:

• To introduce Generalized Discriminant Analysis (GerDA) as a generalization of LDA using deep neural networks (DNNs).
• To enable nonlinear transformations for improved feature extraction in semi-supervised learning settings.
• To demonstrate the efficacy of GerDA for real-world recognition and detection tasks.

Main Methods:

• Developed GerDA, a novel approach leveraging DNNs for semi-supervised learning of nonlinear transformations.
• Assumed independent Gaussian class conditionals for discriminative feature generation.
• Evaluated GerDA features for dimensionality reduction, visualization, classification, and detection.

Main Results:

• GerDA demonstrated excellent performance on real-world tasks, including handwritten digit recognition and face detection.
• GerDA effectively preprocesses high-dimensional data into low-dimensional representations.
• These low-dimensional representations facilitate accurate predictions using simple linear models or similarity measures.

Conclusions:

• GerDA, powered by DNNs, offers a robust generalization of LDA for enhanced feature extraction.
• The method successfully addresses limitations of linear transformations in handling complex data distributions.
• GerDA provides a powerful tool for high-dimensional data preprocessing, improving downstream prediction accuracy.