Deep Canonical Time Warping for Simultaneous Alignment and Representation Learning of Sequences

Area of Science:

• Machine Learning
• Time-Series Analysis
• Data Science

Background:

• Accurate temporal alignment is critical for machine learning models analyzing time-series data.
• Existing methods often fail due to reliance on linear projections or observation space, limiting capture of complex, non-linear representations.
• This limitation is particularly pronounced when dealing with multi-modal data, such as visual and acoustic information.

Purpose of the Study:

• To introduce Deep Canonical Time Warping (DCTW), a novel method for automatic non-linear representation learning of multiple time-series.
• To achieve both maximal correlation in a shared subspace and temporal alignment of the time-series.
• To extend DCTW to a supervised setting, leveraging labels to enhance the alignment process.

Main Methods:

• Developed Deep Canonical Time Warping (DCTW) to learn hierarchical, non-linear representations.
• DCTW maximizes correlation between time-series in a shared subspace while ensuring temporal alignment.
• An extension incorporates supervised learning using available labels to refine alignment.

Main Results:

• DCTW demonstrated superior performance in temporal alignment compared to state-of-the-art methods across four datasets.
• The learned representations effectively handle heterogeneous feature sets.
• Successfully demonstrated accurate temporal alignment of acoustic and visual information.

Conclusions:

• Deep Canonical Time Warping (DCTW) offers a powerful approach for non-linear temporal alignment of multi-series data.
• The method excels in scenarios involving multi-modal and heterogeneous data.
• DCTW provides a significant advancement for time-series analysis applications requiring robust temporal alignment.