Benchmarking criteria to determine latent linear dimensionality in neural data

Dimensionality reduction is widely used in modern Neuroscience to process massive neural recordings data. Despite the development of complex non-linear techniques, linear algorithms, in particular principal component analysis (PCA), are still the gold standard. However, there is no consensus on how to estimate the optimal number of latent variables to retain. In this study, we addressed this issue by testing different criteria on simulated data. Parallel analysis, optimal singular value thresholding and cross-validation proved to be the best methods, being largely unaffected by the number of units and the amount of noise. Notably, among the CV schemes tested, cross-validating along either feature and observation dimensions simultaneously provided the best performance, whereas cross-validating only along one matrix dimension was suboptimal. In addition, we showed that given a data matrix with known properties (such as the size and the decay rate of the eigenvalues) the number of significant components that can be estimated is extremely limited. Finally, as an example application to real neural data, we analyzed the spiking activity of a publicly available dataset recorded from macaque primary visual cortex. We show that different criteria can lead to remarkably different results, whereas most of them still identify similar trends in the estimated dimensionality. Our findings suggest that the term 'dimensionality' needs to be defined carefully and, more importantly, that the most robust criteria for choosing the number of dimensions should be adopted in future works. To help other researchers with the implementation of such an approach on their data, we provide a simple software package, and we present the results of our simulations through a simple Web based app to guide the choice of latent variables in a variety of new studies.