Feature Import Vector Machine: A General Classifier with Flexible Feature Selection.

Area of Science:

• Machine Learning
• Computational Statistics

Background:

• Support Vector Machines (SVMs) and Reproducing Kernel Hilbert Space (RKHS) classifiers are recognized for robustness and generalization.
• Traditional SVMs utilize all dimensions, which may be suboptimal for high-dimensional, small-sample datasets (large p, small n).

Purpose of the Study:

• To develop an algorithmic approach for optimal feature selection in high-dimensional classification.
• To address the challenge where not all features contribute equally to classification accuracy.

Main Methods:

• Reversing the traditional SVM observation selection to a sequential feature selection strategy.
• Modifying the Zhu and Hastie (2005) solution within the Import Vector Machine (IVM) framework.
• Developing an algorithm to identify an optimal subset of features.

Main Results:

• The proposed method enables selection of an optimal feature subset, leading to potential data compression.
• Achieves sufficient accuracy by building a classifier on a reduced sub-dimensional model.
• Demonstrates an effective alternative to using all available dimensions in high-dimensional spaces.

Conclusions:

• The sequential feature selection approach offers an effective strategy for optimizing SVMs in high-dimensional settings.
• This method enhances data compression and classification accuracy by focusing on relevant features.
• The algorithm provides a robust solution for constructing accurate classifiers from large-p, small-n data.