A new hybrid filter/wrapper algorithm for feature selection in classification.

Area of Science:

• Machine Learning
• Data Science
• Bioinformatics

Background:

• High-dimensional datasets pose challenges for learning algorithms.
• Existing feature selection methods (filter and wrapper) have limitations in accuracy and computational cost.

Purpose of the Study:

• Introduce a novel hybrid feature selection method, the large margin hybrid algorithm for feature selection (LMFS).
• Address limitations of existing feature selection techniques.
• Enhance classification performance and model interpretability for high-dimensional data.

Main Methods:

• Utilize a novel distance-based evaluation function for feature subset candidate generation.
• Employ a weighted bootstrapping search strategy.
• Incorporate a specific classifier and cross-validation for final subset selection.

Main Results:

• LMFS effectively mitigates overfitting between feature subsets and classifiers.
• Features selected by LMFS demonstrate superior classification performance and model interpretability compared to filter and wrapper methods.
• LMFS is robust to classifier complexity, with distance-based classifiers proving most suitable for final subset selection.

Conclusions:

• LMFS presents a powerful and efficient feature selection approach for high-dimensional data.
• The method enhances predictive accuracy and provides better insights into the data.
• LMFS offers a valuable alternative to conventional feature selection techniques in machine learning applications.