A greedy feature selection algorithm for Big Data of high dimensionality.

Area of Science:

• Machine Learning
• Computational Statistics
• Bioinformatics

Background:

• High-dimensional Big Data presents significant challenges for traditional feature selection methods.
• Scalability and computational efficiency are critical for analyzing large datasets.

Purpose of the Study:

• To develop a massively parallel algorithm for feature selection (FS) in high-dimensional Big Data.
• To introduce the Parallel, Forward-Backward with Pruning (PFBP) algorithm designed for efficiency and scalability.

Main Methods:

• PFBP partitions data matrices by rows and columns, utilizing local computations and meta-analysis.
• Employs p-values of conditional independence tests to minimize communication costs.
• Incorporates heuristics like Early Dropping, Early Stopping, and Early Return for efficient decision-making.

Main Results:

• PFBP demonstrates super-linear speedup with increasing sample size and linear scalability with features and cores.
• Empirical analysis confirms its effectiveness and superior performance compared to other feature selection algorithms.
• Asymptotic guarantees of optimality are provided for data representable by causal networks.

Conclusions:

• PFBP is a highly scalable and efficient algorithm for feature selection in Big Data.
• Its parallel architecture and heuristics enable effective analysis of high-dimensional datasets.
• The presented heuristics offer potential improvements for other greedy feature selection algorithms.