Weighted mahalanobis distance kernels for support vector machines.

Area of Science:

• Machine Learning
• Computer Vision
• Pattern Recognition

Background:

• Support Vector Machines (SVMs) are effective classifiers but underutilize data distribution information for decision hyperplane determination.
• Existing nonlinear SVM kernels often rely on Euclidean distance, treating SVMs as 'local' classifiers and ignoring broader data distribution tendencies.
• This limits SVM performance by not fully leveraging the inherent structure within datasets.

Purpose of the Study:

• To develop a novel kernel approach for SVMs that incorporates data-specific knowledge, moving beyond traditional similarity measures.
• To enhance SVM classification by integrating data distribution information directly into kernel construction.
• To improve the performance of SVMs by creating more informative kernel functions.

Main Methods:

• Adaptive data structure identification for each class using agglomerative hierarchical clustering (AHC) in the input space.
• Construction of weighted Mahalanobis distance (WMD) kernels utilizing the identified data distribution.
• WMD kernels measure similarity based on Mahalanobis distance and cluster sizes, operating in pseudo-Euclidean (pE) spaces.

Main Results:

• Experimental results demonstrate the effectiveness of WMD kernels on both synthetic and real-world datasets.
• The proposed method shows improved classification performance by incorporating data structure into existing kernels.
• Satisfactory classification outcomes were achieved despite WMD kernels not being guaranteed positive definite (pd) or conditionally positive definite (cpd).

Conclusions:

• Incorporating data-specific knowledge, such as cluster structure, into kernel functions significantly enhances SVM classification.
• WMD kernels offer a promising approach to 'plugging' data structure into existing kernel paradigms for improved machine learning performance.
• The study validates the effectiveness of using data distribution information for more robust and accurate pattern recognition.