A Novel Formulation of Trace Ratio Linear Discriminant Analysis

Area of Science:

• Machine Learning
• Pattern Recognition
• Data Science

Background:

• Traditional Linear Discriminant Analysis (LDA) methods often yield approximate solutions, introducing errors.
• Existing dimensionality reduction (DR) techniques exhibit sensitivity to the choice of subspace dimensionality.

Purpose of the Study:

• To propose a new formulation of Trace Ratio LDA (TRLDA) offering an optimal solution for LDA.
• To introduce Optimal Dimensionality LDA (ODLDA) for determining the optimal subspace dimension in DR.

Main Methods:

• Developed a novel formulation of Trace Ratio LDA (TRLDA).
• Transformed the TRLDA projection matrix solution into a quadratic problem on the Stiefel manifold.
• Proposed Optimal Dimensionality LDA (ODLDA) based on a trace difference problem with guaranteed optimal subspace dimensionality.

Main Results:

• The proposed TRLDA provides an optimal solution for LDA, unlike approximate methods.
• ODLDA's nonmonotonicity ensures the existence of an optimal subspace dimension.
• Both TRLDA and ODLDA demonstrated efficient dimensionality reduction across various datasets.

Conclusions:

• TRLDA offers a more accurate and optimal approach to LDA.
• ODLDA effectively identifies optimal subspace dimensions, enhancing DR performance.
• The proposed methods significantly advance dimensionality reduction techniques.