Feature Selection With Maximal Relevance and Minimal Supervised Redundancy.

Area of Science:

• Machine Learning
• Bioinformatics
• Data Science

Background:

• High-dimensional data, common in image and bio-microarray analysis, presents challenges for machine learning due to irrelevant and redundant features.
• Ineffective feature selection can degrade classifier performance and lead to misclassification.
• Existing feature selection methods may not adequately address both feature relevance and inter-feature redundancy in relation to class labels.

Purpose of the Study:

• To develop an efficient feature selection (FS) algorithm for improving classification accuracy in high-dimensional datasets.
• To propose a novel supervised similarity measure based on conditional mutual information and entropy.
• To introduce and theoretically validate a new criterion, max-relevance and min-supervised-redundancy (MRMSR), for feature selection.

Main Methods:

• A novel supervised similarity measure was developed using conditional mutual information and entropy.
• This measure was integrated with evaluations for feature redundancy minimization and feature relevance maximization.
• The proposed max-relevance and min-supervised-redundancy (MRMSR) criterion was theoretically established for feature selection.

Main Results:

• The MRMSR-based feature selection method was compared against seven established FS approaches.
• Experiments were conducted on multiple public benchmark datasets commonly used in the field.
• The proposed method demonstrated superior effectiveness in selecting informative features compared to existing techniques.

Conclusions:

• The MRMSR algorithm offers a more effective approach to feature selection for classification tasks.
• The method achieves competitive and improved classification performance by optimizing feature selection.
• This work contributes a robust feature selection strategy for handling large-scale, high-dimensional data in machine learning.