Utilizing stability criteria in choosing feature selection methods yields reproducible results in microbiome data

Area of Science:

• Microbiome data analysis
• Bioinformatics
• Computational biology

Background:

• Microbiome datasets are high-dimensional, sparse, and compositional, posing challenges for feature selection.
• Existing feature selection methods are often evaluated solely on prediction accuracy, neglecting reproducibility.
• This overlooks the potential for selected features to be data artifacts rather than true biological signals.

Purpose of the Study:

• To critically evaluate current feature selection criteria in microbiome analysis.
• To compare the performance of model prediction metrics (MSE, AUC) against a reproducibility criterion (Stability).
• To determine a more appropriate evaluation metric for robust feature selection in microbiome studies.

Main Methods:

• Evaluated four popular feature selection methods.
• Compared model prediction metrics (Mean Squared Error, Area Under the Curve) with the Stability criterion.
• Utilized both simulation studies and experimental microbiome data with continuous and binary outcomes.

Main Results:

• Model prediction metrics may not adequately assess the reliability of selected features.
• Stability criterion quantifies the robustness of feature selection methods to data perturbations.
• Feature selection methods showed varying performance when evaluated by prediction accuracy versus Stability.

Conclusions:

• Stability is a superior criterion for evaluating feature selection in microbiome analysis compared to prediction accuracy.
• The Stability metric better reflects the reproducibility and biological relevance of selected features.
• This work advocates for the adoption of Stability in assessing feature selection methods for microbiome data.