Consensus features nested cross-validation.

Area of Science:

• Machine learning
• Computational biology
• Bioinformatics

Background:

• Feature selection is crucial for machine learning model accuracy but risks overfitting.
• Nested cross-validation (nCV) and differential privacy are existing methods to mitigate overfitting.
• nCV selects features based on inner-fold accuracy, while differential privacy uses noise for feature stability.

Purpose of the Study:

• Introduce consensus nested cross-validation (cnCV), a novel method combining feature stability and nCV.
• Evaluate cnCV's performance against standard nCV, Elastic Net, differential privacy, and private evaporative cooling (pEC).
• Assess cnCV using simulated data and real RNA-seq data for major depressive disorder.

Main Methods:

• Developed cnCV, using feature consensus across inner folds for stability instead of accuracy.
• Compared cnCV with nCV, Elastic Net, differential privacy, and pEC on simulated and real datasets.
• Analyzed classification accuracy, feature selection performance, and computational efficiency.

Main Results:

• cnCV achieved similar training and validation accuracy to nCV but with significantly reduced run times.
• cnCV identified a more parsimonious set of features with fewer false positives compared to nCV.
• cnCV demonstrated comparable accuracy to pEC and effectively selected stable features without requiring a privacy threshold.

Conclusions:

• cnCV is an effective and efficient approach for integrating feature selection with classification.
• The method offers a balance of accuracy, efficiency, and parsimonious feature selection.
• cnCV provides a robust alternative for feature selection in machine learning, particularly in bioinformatics.