Normalized mutual information feature selection.

Area of Science:

• Machine Learning
• Data Mining
• Bioinformatics

Background:

• Feature selection is crucial for improving model performance and interpretability in high-dimensional datasets.
• Existing methods like MIFS, MIFS-U, and mRMR have limitations in handling feature redundancy and parameter tuning.
• Mutual information is a powerful metric for quantifying feature dependencies.

Purpose of the Study:

• To introduce Normalized Mutual Information Feature Selection (NMIFS), an enhanced filter method for feature selection.
• To propose average normalized mutual information as a measure for feature redundancy.
• To develop a hybrid genetic algorithm and NMIFS (GAMIFS) for improved feature subset identification.

Main Methods:

• Developed NMIFS, a filter method utilizing normalized mutual information to assess feature relevance and redundancy.
• Introduced average normalized mutual information to quantify inter-feature redundancy.
• Integrated NMIFS into a genetic algorithm (GAMIFS) with a specialized initialization and mutation operator.

Main Results:

• NMIFS demonstrated superior performance compared to MIFS, MIFS-U, and mRMR on artificial and benchmark datasets.
• NMIFS effectively reduced feature redundancy without requiring user-defined parameters.
• GAMIFS accelerated genetic algorithm convergence and overcame limitations of incremental search methods.

Conclusions:

• NMIFS offers a robust and parameter-free feature selection method.
• The hybrid GAMIFS approach effectively identifies feature dependencies, including group interactions.
• NMIFS and GAMIFS represent significant advancements in feature selection techniques for complex datasets.