Time-series benchmarks based on frequency features for fair comparative evaluation.

Area of Science:

• Machine Learning
• Time-Series Analysis
• Signal Processing

Background:

• Time-series prediction and imputation are critical in academia and industry.
• Existing machine learning methods are often scenario-specific, hindering cross-case evaluation.
• A standardized benchmark is needed to assess the generalizability of time-series models.

Purpose of the Study:

• To develop a comprehensive benchmark dataset for evaluating time-series prediction methods.
• To enable fair and reproducible assessment of machine learning algorithms across diverse scenarios.
• To introduce a frequency-domain perspective for time-series problem generation.

Main Methods:

• A novel prediction problem generation process using finite impulse response (FIR) filters.
• The NCAA2022 dataset generation, comprising 16 distinct prediction problems.
• Application of Discrete Fourier Transform (DFT) for frequency analysis of time-series data.
• Matrix decomposition to optimize computational efficiency for filter parameters.

Main Results:

• The NCAA2022 dataset provides a robust benchmark for time-series prediction.
• Frequency analysis reveals insights into the characteristics of generated time-series problems.
• A baseline experiment demonstrates the dataset's utility in evaluating prediction models.
• The method allows for efficient generation and analysis of complex time-series tasks.

Conclusions:

• The NCAA2022 dataset offers a valuable resource for advancing time-series prediction research.
• Frequency-based analysis provides a powerful lens for understanding and generating time-series problems.
• The developed benchmark facilitates more reliable evaluation of machine learning models in this domain.