A Distributed Feature Selection Algorithm Based on Distance Correlation with an Application to Microarrays.

Area of Science:

• Bioinformatics
• Computational Biology
• Machine Learning

Background:

• DNA microarray datasets present challenges due to high dimensionality (many features) and low sample size, leading to overfitting and poor generalization in classification.
• Existing feature selection methods may struggle with the complex dependencies and redundancy present in such high-dimensional biological data.

Purpose of the Study:

• To develop a novel feature selection (FS) approach for DNA microarray data that addresses overfitting and enhances classification performance.
• To utilize distance correlation (dCor) as a robust measure of feature-class dependence, sensitive to non-linear relationships and redundancy.

Main Methods:

• A new feature selection method employing distance correlation (dCor) as the criterion for evaluating feature subset dependence on the class label.
• The method uses a probabilistic model representation, refined through iterative model extraction and evaluation.
• A distributed optimization scheme with vertical data partitioning is implemented to handle unbalanced dataset dimensions effectively.

Main Results:

• The proposed FS method, utilizing dCor, demonstrated effectiveness in selecting relevant features from DNA microarray datasets.
• Tested on multiple microarray datasets, the approach yielded compact and accurate classification models.
• The method achieved these results at a reasonable computational cost.

Conclusions:

• The novel feature selection approach based on distance correlation is a promising technique for improving classification of DNA microarray data.
• It effectively mitigates overfitting and enhances model generalization by selecting informative and non-redundant features.
• The method offers a computationally efficient solution for high-dimensional biological data analysis.