A Fusion Transformer for Multivariable Time Series Forecasting: The Mooney Viscosity Prediction Case

Area of Science:

• Machine Learning
• Data Science
• Materials Science

Background:

• Multivariable time series forecasting often requires integrating diverse data types, including static covariates and exogenous time series.
• Improving prediction performance necessitates targeted investigation and effective fusion of these varied input data.
• Existing methods may struggle to optimally combine static and dynamic information for complex forecasting tasks.

Purpose of the Study:

• To propose and evaluate the Fusion Transformer (FusFormer), a novel transformer-based model for multivariable time series forecasting.
• To demonstrate FusFormer's capability in fusing time series data and static covariates for enhanced prediction.
• To validate the model's effectiveness and interpretability using a case study in Mooney viscosity forecasting.

Main Methods:

• Developed FusFormer, a transformer-based architecture with parallel processing stages for time series and static data.
• Employed a temporal encoder-decoder framework to extract dynamic features and integrate positional information via attention mechanisms.
• Utilized a static enrichment module, inspired by gated linear units, to process static covariates, suppress noise, and control nonlinearity.

Main Results:

• FusFormer achieved significant forecasting performance improvements over existing methodologies in Mooney viscosity prediction.
• Ablation analysis confirmed the effectiveness of individual components within the FusFormer architecture.
• An interpretability use case successfully visualized temporal patterns, demonstrating the model's ability to capture time series dynamics.

Conclusions:

• FusFormer effectively fuses multivariable time series data and static covariates, leading to superior forecasting accuracy.
• The model's design enhances the extraction and integration of dynamic temporal features and static information.
• The application to Mooney viscosity forecasting shows potential for improving industrial processes, such as tire production efficiency.