Novel Insights on Induced Sparsity in Multi-Time Attention Networks

Current deep learning approaches for dealing with sparse irregularly sampled time-series data do not exploit the extent of sparsity of the input data. Our work is inspired by the sparse and irregularly sampled nature of physiological time series data in electronic health records. We explore the effect of inducing varying degrees of sparsity on the predictive performance of Multi-Time Attention Networks (mTAN) [1]. Our methodology is to induce sparsity by first sub-sampling the time-series before feeding it to the mTAN network. We conduct empirical experiments with sub-sampling ranging from 10 to 90 %. We investigate the performance of our methodology on the Human Activity dataset and Physionet 2012 mortality prediction task. Our results demonstrate that our proposed time-point sub-sampling coupled with mTAN improves the performance by 2 % on the Human Activity dataset with 80 % lesser time-points for training. On the Physionet dataset, our approach achieves comparable performance as baseline with 30 % lesser time-points. Our experiments reveal that time-series data could be further coarsely acquired when used in tandem with state-of-the-art networks capable of handling sparse data (mTAN). This could be of immense help for various applications where data acquisition and labeling is a significant challenge.