Multiclass feature selection with kernel Gram-matrix-based criteria

Area of Science:

• Machine Learning
• Data Science
• Computational Statistics

Background:

• Feature selection is crucial for classification tasks, aiming to identify the most relevant data attributes.
• Existing Support Vector Machine (SVM)-based feature selection methods are often complex and computationally expensive.
• There is a need for efficient and scalable feature selection techniques for SVMs, especially for multiclass problems.

Purpose of the Study:

• To introduce novel feature selection methods for Support Vector Machines (SVMs).
• To leverage kernel target alignment and kernel class separability for SVM optimization.
• To develop methods that are efficient, simple, and suitable for multiclass classification with minimal memory footprint.

Main Methods:

• Proposed feature selection methods based on kernel target alignment and kernel class separability criteria.
• Iterative computation of feature relevance with minimal memory requirements.
• Application to multiclass classification problems.

Main Results:

• The proposed methods demonstrate efficiency and simplicity in feature selection for SVMs.
• Experimental results on artificial and real-world datasets show competitive performance compared to state-of-the-art algorithms.
• The methods offer a favorable balance between classification performance and computational cost.

Conclusions:

• The novel feature selection methods based on kernel target alignment and kernel class separability are effective and efficient.
• These methods provide a valuable alternative for optimizing SVMs, particularly in multiclass scenarios.
• The proposed approach reduces computational complexity while maintaining or improving classification accuracy.