Biobjective gradient descent for feature selection on high dimension, low sample size data | JoVE Visualize

Area of Science:

Machine Learning
Artificial Intelligence
Computational Biology

Background:

Deep learning models often overfit when applied to High Dimensions and Low Sample Size (HDLSS) data, common in rare disease diagnosis.
Traditional solutions like feature selection and network sparsification are typically addressed independently.
This limitation hinders the effective application of deep learning in critical areas like medical diagnostics.

Purpose of the Study:

To propose a novel approach that integrates feature selection and network sparsification within the deep neural network training process.
To develop a method that optimizes the trade-off between network sparsity and model accuracy for HDLSS data.
To enhance the performance of deep learning models in scenarios characterized by limited data and numerous features.

Main Methods:

A constrained biobjective gradient descent method was developed to simultaneously optimize for feature selection and network sparsity.
The approach integrates feature selection directly into the deep neural network's training, treating it as a sparsification problem.
The method generates a set of Pareto optimal neural networks, offering a spectrum of trade-offs between sparsity and accuracy.

Main Results:

The integrated approach significantly increased network sparsity (0.92) and feature selection scores (0.97) on an artificial dataset, while maintaining high accuracy (0.9).
Compared to other methods, the proposed approach achieved substantially higher feature selection and sparsity scores at equivalent accuracy levels.
Statistical validation confirmed the effectiveness of the constrained biobjective gradient descent method across both artificial and real-world datasets.

Conclusions:

Integrating feature selection via sparsification into deep neural network training is an effective strategy for HDLSS data.
The constrained biobjective gradient descent method successfully balances network sparsity and classification accuracy.
This unified approach offers a significant advancement over separate feature selection and sparsification techniques for challenging datasets.