A Regularized Linear Dynamical System Framework for Multivariate Time Series Analysis.

Area of Science:

• Machine Learning
• Time Series Analysis
• Dynamical Systems

Background:

• Linear Dynamical Systems (LDS) are effective for Multivariate Time Series (MTS) but selecting the correct hidden state dimension is challenging.
• An incorrect dimension can lead to underfitting or overfitting, hindering accurate modeling and forecasting of MTS data.

Purpose of the Study:

• To develop a regularized LDS (rLDS) learning framework to automatically identify and eliminate unnecessary hidden state dimensions.
• To address the overfitting problem in MTS data with limited samples.
• To improve the accuracy of MTS forecasting using a robust LDS model.

Main Methods:

• Incorporation of regularization penalties on the LDS transition matrix.
• Utilizing a second-order cone program and generalized gradient descent within a Maximum a Posteriori framework.
• Employing Expectation Maximization to achieve a low-rank transition matrix for the LDS model.

Main Results:

• The rLDS framework successfully recovers the intrinsic dimensionality of time series dynamics.
• Demonstrated improvement in predictive performance compared to baseline methods on both synthetic and real-world MTS datasets.
• Effective prevention of overfitting, even with small MTS datasets.

Conclusions:

• The proposed rLDS framework offers an effective solution for determining optimal hidden state dimensions in LDS models for MTS.
• rLDS enhances the accuracy and robustness of MTS forecasting by mitigating overfitting and accurately capturing underlying dynamics.
• This approach provides a significant advancement in learning and modeling complex multivariate time series data.