Asymptotic Generalization Bound of Fisher's Linear Discriminant Analysis

Area of Science:

• Statistical Pattern Recognition
• Machine Learning Theory

Background:

• Fisher's linear discriminant analysis (FLDA) is asymptotically Bayes optimal under homoscedastic Gaussian assumptions.
• Classical FLDA results are limited to fixed dimensionality and lack quantitative generalization analysis for large D and N.

Purpose of the Study:

• To present an asymptotic generalization analysis of FLDA using random matrix theory.
• To overcome limitations of classical FLDA results for proportionally large dimensions (D) and sample sizes (N).

Main Methods:

• Asymptotic analysis of FLDA generalization.
• Application of random matrix theory for large D and N settings where D/N approaches a constant gamma.
• Derivation of a lower bound for generalization discrimination power.

Main Results:

• A novel lower bound for FLDA generalization discrimination power is established, valid for proportionally large D and N (D/N → γ).
• The analysis provides a quantitative description of FLDA's generalization ability in terms of the ratio γ and population discrimination power.
• An upper bound on the generalization error for binary classification with FLDA is derived.

Conclusions:

• The random matrix theory-based analysis extends FLDA's applicability and provides crucial insights into its generalization performance.
• The findings offer a more comprehensive understanding of FLDA in high-dimensional and large-sample scenarios.
• This work enhances the theoretical foundation for using FLDA in modern statistical pattern recognition.