Discrete Representation Learning for Multivariate Time Series

Area of Science:

• Machine Learning
• Time Series Analysis
• Computational Neuroscience

Background:

• Multivariate time series analysis often faces challenges with high dimensionality and interpretability.
• Deep learning models struggle with incorporating discrete latent variables due to non-differentiability issues.
• Gaussian processes offer a probabilistic framework valuable for time series modeling.

Purpose of the Study:

• To develop a novel deep learning architecture for discrete representation learning in multivariate time series.
• To enhance the interpretability of time series data by learning low-dimensional embeddings and discrete latent states.
• To improve classification performance on complex time series datasets, such as fMRI data.

Main Methods:

• Utilized a Gumbel-softmax reparameterization trick to handle non-differentiability when integrating discrete latent variables.
• Developed a joint clustering and embedding framework through learnable discretization of the latent space.
• Employed Gaussian processes within the deep learning architecture for time series modeling.

Main Results:

• Successfully enabled joint learning of embeddings and discrete latent states for multivariate time series.
• Demonstrated enhanced interpretability by reducing dimensionality and identifying distinct latent states.
• Achieved improved classification results on both synthetic and real-world fMRI datasets.

Conclusions:

• The proposed discrete representation learning method effectively addresses challenges in deep learning for time series.
• The model provides a more interpretable representation of complex time series data.
• The approach shows significant potential for applications in neuroimaging and other fields involving high-dimensional time series data.