Large Scale Time-Series Representation Learning via Simultaneous Low- and High-Frequency Feature Bootstrapping.

Area of Science:

• Machine Learning
• Time Series Analysis
• Deep Learning

Background:

• Learning representations from unlabeled time series data is challenging.
• Existing methods struggle to capture both low- and high-frequency features simultaneously, limiting generalization.
• Many current approaches use large models or computationally expensive techniques like contrastive learning.

Purpose of the Study:

• To propose an efficient, noncontrastive self-supervised learning (SSL) approach for time series representation learning.
• To effectively capture both low- and high-frequency features in a cost-effective manner.
• To improve the generalization ability of learned representations.

Main Methods:

• A Siamese neural network configuration with two weight-sharing branches.
• Low- and high-frequency feature extraction modules using Multilayer Perceptron (MLP) and Temporal Convolutional Network (TCN) heads.
• Input data augmentation using random transformations from a single set.

Main Results:

• The proposed method efficiently captures low- and high-frequency features.
• Achieved state-of-the-art performance on five real-world time-series datasets.
• Demonstrated robustness through extensive experiments and ablation studies.

Conclusions:

• The developed noncontrastive SSL framework effectively learns robust time series representations.
• The method offers a cost-effective solution for capturing multi-frequency features.
• Outperforms existing methods in time series representation learning.