The Fisher-Markov selector: fast selecting maximally separable feature subset for multiclass classification with...

Area of Science:

• Machine Learning
• Pattern Recognition
• Data Science

Background:

• Feature selection is crucial for multiclass classification, especially with high-dimensional data.
• Existing methods struggle with scalability, efficiency, and achieving global optimums.
• High-dimensional data often contains noise, missing values, and outliers, complicating feature selection.

Purpose of the Study:

• To introduce an efficient and scalable feature selection method for multiclass classification.
• To identify globally optimal feature subsets from high-dimensional datasets.
• To address the limitations of existing feature selection techniques.

Main Methods:

• Introduction of the Fisher-Markov selector for identifying discriminating features.
• Formulation of an optimization objective incorporating sparsity and discriminativeness.
• Application of Markov random field optimization techniques for simultaneous feature selection.

Main Results:

• The Fisher-Markov selector achieves exact global optimums for specific kernels.
• The method demonstrates efficiency, with linear time complexity in the number of features and quadratic in the number of observations.
• Experimental validation on diverse datasets, including handwritten digits and gene expression data, confirms effectiveness.

Conclusions:

• The Fisher-Markov selector provides an efficient and globally optimal solution for feature selection in high-dimensional multiclass classification.
• The method simplifies the selection process by solving an unconstrained objective function.
• This approach enhances pattern recognition and model selection capabilities.