A kernel-induced space selection approach to model selection in KLDA.

Area of Science:

• Machine Learning
• Data Science
• Pattern Recognition

Background:

• Model selection is crucial for Kernel Linear Discriminant Analysis (KLDA), involving parameter tuning for kernel functions and regularizers.
• Existing methods like cross-validation can be computationally intensive, especially with large datasets or numerous parameters.

Purpose of the Study:

• To develop a computationally efficient and effective model selection approach for KLDA.
• To formulate model selection as identifying an optimal kernel-induced space that maximizes class separability.

Main Methods:

• A novel scatter-matrix-based criterion is proposed to evaluate kernel-induced spaces.
• Kernel parameters are optimized by maximizing this criterion, which is differentiable for efficient tuning.
• The criterion is integrated with Saadi's (2004) method for regularizer parameter tuning.

Main Results:

• The proposed criterion offers a faster model selection compared to leave-one-out (LOO) or k-fold cross-validation (CV).
• The method demonstrates computational efficiency, particularly for large training sets and extensive parameter tuning.
• Experimental results on benchmark datasets validate the effectiveness of the developed model selection approach.

Conclusions:

• The scatter-matrix-based criterion provides an efficient and effective method for KLDA model selection.
• This approach enhances the speed and practicality of tuning KLDA parameters.
• The findings suggest a significant improvement in model selection for KLDA applications.