Ratio Sum Versus Sum Ratio for Linear Discriminant Analysis

Area of Science:

• Machine Learning
• Data Science
• Computer Vision

Background:

• Linear Discriminant Analysis (LDA) is a supervised dimension reduction technique for high-dimensional data.
• Standard LDA may retain features with weak discriminative ability due to its preference for smaller variance features.

Purpose of the Study:

• To propose a novel method, Ratio Sum for Linear Discriminant Analysis (RSLDA), to maximize the discriminative ability of each feature in the subspace.
• To enhance feature selection by focusing on the ratio of between-class to within-class distance.

Main Methods:

• Developed an equivalent problem for RSLDA to enable an alternative optimization algorithm.
• Transformed the problem into two sub-problems, one solved directly and the other as a convex optimization problem using singular value decomposition.
• Introduced Kernel RSLDA (KRSLDA) for improved robustness.

Main Results:

• The proposed RSLDA method effectively maximizes the discriminative ability of features.
• Singular value decomposition avoids issues with non-singular covariance matrices, enhancing algorithmic performance.
• RSLDA and KRSLDA demonstrated superior performance compared to existing methods on various datasets.

Conclusions:

• RSLDA offers a significant improvement over traditional LDA by focusing on feature discriminative power.
• KRSLDA provides a robust extension, suitable for complex data patterns.
• The proposed methods are effective for dimension reduction and feature selection in high-dimensional data processing.