Related Experiment Videos
Feature scaling for kernel fisher discriminant analysis using leave-one-out cross validation.
Liefeng Bo1, Ling Wang, Licheng Jiao
1Institute of Intelligent Information Processing, Xidian University, Xi'an 710071, China. blf0218@163.com
Neural Computation
|February 24, 2006
Summary
A new algorithm, Feature-Scaling Kernel Fisher Discriminant Analysis (FS-KFD), optimizes kernel parameters for improved classification accuracy. This method addresses key challenges in Kernel Fisher Discriminant Analysis by tuning scaling and regularization parameters effectively.
Area of Science:
- Machine Learning
- Pattern Recognition
- Data Mining
Background:
- Kernel Fisher Discriminant Analysis (KFD) is a powerful classification technique.
- Parameter selection (kernel and regularization) is a critical challenge in KFD.
- Feature scaling is often crucial for effective machine learning model performance.
Purpose of the Study:
- To develop a novel algorithm, FS-KFD, for optimizing feature-scaling kernels in KFD.
- To address the challenge of tuning scaling factors and regularization parameters.
- To enhance the classification accuracy of KFD.
Main Methods:
- Developed FS-KFD algorithm for feature-scaling kernel optimization.
- Employed a gradient-descent method to optimize smooth leave-one-out error.
- Utilized closed-form leave-one-out error expression and sigmoid approximation of step functions.
Main Results:
- FS-KFD demonstrated computational feasibility.
- Empirical comparisons showed improved classification accuracy compared to standard KFD.
- The algorithm effectively tunes scaling factors and regularization parameters.
Conclusions:
- FS-KFD offers an effective solution for parameter tuning in feature-scaling KFD.
- The proposed method enhances classification performance on various datasets.
- FS-KFD represents a significant advancement in Kernel Fisher Discriminant Analysis.