Higher criticism thresholding: Optimal feature selection when useful features are rare and weak

Area of Science:

• Statistical Learning
• Bioinformatics
• Machine Learning

Background:

• Effective feature selection is critical for linear classification analysis in fields like genomics and proteomics.
• Traditional methods may struggle with rare/weak features, where useful signals are subtle and infrequent.
• Existing threshold selection procedures like False Discovery Rate Thresholding (FDRT) have limitations in challenging scenarios.

Purpose of the Study:

• To introduce and evaluate Higher Criticism thresholding (HCT) for feature selection in linear classification.
• To compare HCT with existing methods, particularly FDRT, under rare/weak (RW) feature models.
• To assess the impact of HCT on classifier performance, error rates, and threshold selection stability.

Main Methods:

• Feature selection via thresholding of Z-scores based on the Higher Criticism (HC) objective function.
• The HC threshold maximizes (i/p - pi((i)))/sqrt{i/p(1-i/p)}, where pi((i)) are ordered P-values.
• Comparison of HCT with FDRT and cross-validation in Shrunken Centroid classifiers using real datasets and asymptotic theory.

Main Results:

• HCT demonstrates an intimate link between maximizing the HC objective and minimizing classifier error rates in RW settings.
• HCT utilizes lower thresholds than FDRT in challenging RW scenarios, improving control over missed features.
• Replacing cross-validation with HCT in Shrunken Centroid classifiers reduces threshold variance and misclassification error.

Conclusions:

• Higher Criticism thresholding (HCT) provides a computationally efficient and simpler alternative for feature selection.
• HCT offers superior performance over FDRT in rare/weak feature models, leading to more accurate classifiers.
• The study confirms the advantages of HCT in both theoretical analysis and practical applications on real datasets.