Extremely high-dimensional feature selection via feature generating samplings.

Area of Science:

• Machine Learning
• Data Science
• Computational Statistics

Background:

• Feature selection is crucial for high-dimensional data analysis.
• Existing feature generating machines (FGMs) face computational challenges with extremely large feature sets (m > 10^11).
• The time complexity of ordering features in FGMs (O(mlogr)) becomes prohibitive.

Purpose of the Study:

• To propose an efficient sampling scheme for feature generating machines (FGMs).
• To reduce the computational complexity of feature selection in high-dimensional problems.
• To ensure the preservation of the most informative features during the selection process.

Main Methods:

• A feature generating sampling method is proposed.
• The method reduces computational complexity to O(Gslog(G)+G(G+log(G))).
• The sampling scheme is theoretically linked to birth-death processes from random processes theory.

Main Results:

• The proposed method significantly enhances the efficiency of FGMs.
• It effectively preserves the most informative features in a feature buffer.
• Empirical studies on real-world datasets validate the method's effectiveness.

Conclusions:

• The novel sampling scheme offers a computationally efficient solution for feature selection in high-dimensional data.
• The method guarantees the inclusion of most informative features, supported by theoretical foundations.
• This approach is vital for practical applications dealing with massive feature spaces.