A Principled and Data-efficient Information-theoretic Method for Feature Selection.

Area of Science:

• Machine Learning
• Bioinformatics
• Data Science

Background:

• Traditional feature selection methods may miss complex dependencies in mixed-type data.
• Handling continuous and discrete variables simultaneously presents a significant challenge in feature selection.
• Existing approaches can overlook feature redundancy and higher-order dependencies.

Purpose of the Study:

• Introduce kCMI-FS, a novel feature selection method designed for mixed-type data.
• Evaluate the performance of kCMI-FS against existing methods on various datasets.
• Demonstrate the utility of kCMI-FS in enhancing classification accuracy and model interpretability in biomedical applications.

Main Methods:

• Utilized Conditional Mutual Information (CMI) estimated via a k-nearest neighbour (kNN) strategy.
• Implemented a significance-based forward selection process to identify informative and non-redundant features.
• Assessed performance on theoretical simulations, synthetic datasets, and biomedical benchmark datasets.

Main Results:

• kCMI-FS consistently recovered relevant features in structured scenarios.
• The method matched or outperformed existing approaches, especially in mixed-variable and high-dimensional settings.
• Classification experiments showed kCMI-FS achieved strong predictive performance with reduced feature sets, improving interpretability.

Conclusions:

• kCMI-FS is a robust feature selection method for mixed-type data, particularly effective in high-dimensional and complex scenarios.
• The method enhances model interpretability and predictive accuracy in biomedical data analysis.
• kCMI-FS shows significant potential for supporting early diagnosis and clinical decision-making through effective feature selection.