MARINE-Transformer: A General-purpose framework for multivariate ocean time series analysis.

Area of Science:

• Oceanography
• Data Science
• Machine Learning

Background:

• Self-supervised pre-training is successful in text and images but lags for multivariate oceanographic time series.
• Existing methods often use task-specific approaches for forecasting, imputation, and anomaly detection.

Purpose of the Study:

• To propose a general-purpose framework, MARINE-Transformer, for task-agnostic pre-training of oceanographic time series.
• To learn general representations adaptable to downstream tasks via parameter-efficient fine-tuning.
• To address heterogeneous dependencies in ocean data: universal temporal dynamics and specific cross-variable physics.

Main Methods:

• A novel univariate-to-multivariate paradigm is introduced.
• Univariate pre-training uses a Masked AutoEncoder (MAE) to learn intrinsic temporal dynamics of individual variables.
• Multivariate fine-tuning formulates tasks as mask-reconstruction problems, leveraging a pre-trained encoder to build a dependency graph for inter-variable interactions.

Main Results:

• The framework achieves state-of-the-art performance on multiple real-world oceanographic datasets.
• Demonstrated effectiveness in forecasting, data imputation, and anomaly detection tasks.
• Successfully models both universal temporal dynamics and specific cross-variable physics.

Conclusions:

• MARINE-Transformer provides a robust and generalizable approach for multivariate time series analysis in oceanography.
• The proposed univariate-to-multivariate paradigm effectively captures complex dependencies in ocean data.
• This framework offers significant improvements over existing methods for critical oceanographic data analysis tasks.