Comparison of dimensionality reduction and feature selection for cognitive task decoding using functional
Corey J Richier1, Kyle A Baacke2, Sarah M Olshan1
1University of Illinois Urbana-Champaign, Department of Psychology, 603 E Daniel St., Champaign, IL 61820, United States.
Background:
Advances in functional magnetic resonance imaging (fMRI) have led to the ability to study the brain across many contexts. However, the large number of features generated by functional connectivity approaches may overfit the data. These problems can be overcome with either feature selection (FS) or dimensionality reduction (DR), which can be applied to less complex models. We utilize two open source datasets to compare the performance of DR/FS methods on cognitive task decoding using a suite of ML classifiers.
New Method:
While FS and DR methods have been used previously in decoding research, no systematic comparison of their performance has been undertaken. Here, we compare available methods using commonly utilized machine learning libraries to establish which methods provide the best predictive performance. We then conduct statistical tests to examine the relative contributions of DR and FS methods and classifiers on decoding accuracy.
Results:
Neither DR or FS was found to be superior. However, differences were identified across datasets and tasks. In the majority of methods and datasets, a peak in predictive performance was found using a small percentage (005-.10%) of the total number of original features.
Comparison With Existing Methods:
Some methods perform better than the baseline method of prediction with all available features or selecting features randomly. Decoding performance with some datasets with some method exceeds that of deep learning approaches.
Conclusions:
These results suggest a "sweet spot" for the tradeoff between the retention of features and predictive accuracy.
