Deep Least Squares Fisher Discriminant Analysis.

Area of Science:

• Machine Learning
• Data Science
• Pattern Recognition

Background:

• Fisher linear discriminant analysis (FLDA) is an early dimensionality reduction technique.
• FLDA is not currently a leading method for feature extraction or classification.
• Recent advancements include least squares FLDA (LSFDA) and regularized kernel FDA (RKFDA).

Purpose of the Study:

• To introduce Deep FDA (DFDA), a nonlinear extension of LSFDA using deep neural networks.
• To compare the performance of RKFDA and DFDA on various classification tasks.
• To evaluate efficiency, particularly for large and imbalanced datasets.

Main Methods:

• Proposed Deep FDA (DFDA) as a nonlinear extension of LSFDA.
• Utilized deep neural networks for nonlinear feature extraction.
• Compared RKFDA and DFDA using Area Under the ROC Curve (AUC) on diverse datasets.
• Assessed performance on two-class, multiclass, and imbalanced data.

Main Results:

• DFDA and RKFDA demonstrated similar classification performance, particularly on imbalanced datasets.
• DFDA models were significantly faster to build than RKFDA models.
• Both methods showed good performance on problems with large sample sizes.

Conclusions:

• DFDA provides a competitive and computationally efficient alternative to RKFDA.
• DFDA is a promising nonlinear extension of FLDA for classification tasks.
• The proposed method excels in scenarios with class-imbalanced data and large sample sizes.