Simple strategies for semi-supervised feature selection.

Area of Science:

• Machine Learning
• Data Science
• Computational Statistics

Background:

• Semi-supervised learning leverages both labeled and unlabeled data.
• Feature selection is crucial for model efficiency and interpretability.
• Existing methods often require complex assumptions or extensive labeled data.

Purpose of the Study:

• To investigate the efficacy of minimalist, classifier-independent strategies for semi-supervised feature selection.
• To develop novel algorithms based on simple assumptions and domain knowledge.
• To evaluate performance against complex competing methods, particularly in scenarios with missing-not-at-random labels.

Main Methods:

• Theoretical analysis and empirical studies of two simple strategies: assuming unlabeled data are all positive or all negative.
• Utilizing hypothesis testing and feature ranking for feature selection.
• Developing two novel algorithms, Semi-JMI and Semi-IAMB, by incorporating soft prior domain knowledge.

Main Results:

• The simple strategies provide powerful results for feature selection.
• The novel algorithms (Semi-JMI, Semi-IAMB) significantly outperform more complex methods.
• Exceptional performance was observed in cases where labels are missing-not-at-random.

Conclusions:

• Minimalist approaches to semi-supervised feature selection can be surprisingly effective.
• These simple strategies can provably recover exact feature selection dynamics, mimicking a fully labeled dataset.
• The findings suggest a paradigm shift towards simpler, more robust feature selection techniques.