Related Experiment Videos
Efficient and robust feature extraction by maximum margin criterion
Haifeng Li1, Tao Jiang, Keshu Zhang
1Department of Computer Science and Engineering, University of California, Riverside, CA 92521, USA. hli@cs.ucr.edu
IEEE Transactions on Neural Networks
|March 11, 2006
Summary
New feature extraction methods based on the maximum margin criterion (MMC) offer improved class separability and stability over Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA), especially for small sample sizes.
Area of Science:
- Computer Science
- Machine Learning
- Pattern Recognition
Background:
- Feature extraction is crucial for dimensionality reduction and enhancing discriminatory information in pattern recognition.
- Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) are common linear methods, but PCA lacks discriminant power and LDA suffers from instability with small sample sizes.
Purpose of the Study:
- To propose novel linear and nonlinear feature extractors using the Maximum Margin Criterion (MMC).
- To address the limitations of existing methods, particularly the small sample size problem in LDA.
Main Methods:
- Developed new feature extractors based on MMC, which geometrically maximizes the margin between classes post-dimensionality reduction.
- Derived LDA from MMC with constraints and established a new linear feature extractor robust to small sample sizes.
- Introduced a kernelized (nonlinear) version of the new linear feature extractor.
Main Results:
- MMC demonstrates superior class separability compared to PCA.
- The new linear feature extractor effectively overcomes the small sample size problem, enhancing stability.
- Experimental results validate the effectiveness, stability, and efficiency of the proposed feature extractors.
Conclusions:
- The proposed MMC-based feature extractors offer significant advantages over traditional PCA and LDA.
- These new methods provide effective and stable solutions for dimensionality reduction in pattern recognition, particularly in challenging small sample size scenarios.