An investigation into in-sample and out-of-sample model selection for nonstationary autoregressive models

Area of Science:

• Psychology
• Time-Series Analysis
• Statistical Modeling

Background:

• Stationary autoregressive models are foundational in psychology time-series analysis.
• Nonstationary models capture evolving temporal dynamics but lack clear selection guidance.
• Accurate model selection is vital for understanding psychological time-series data.

Purpose of the Study:

• To evaluate in-sample and out-of-sample model selection techniques for nonstationary time-series.
• To compare the performance of different selection methods across simulated nonstationary processes.
• To provide guidance on selecting appropriate nonstationary time-series models in psychological research.

Main Methods:

• A simulation study assessed model selection performance on six univariate nonstationary processes.
• In-sample (information criteria) and out-of-sample (cross-validation, prediction) methods were evaluated.
• A real-world time-series of affect data was re-analyzed to illustrate model selection.

Main Results:

• Bayesian Information Criteria (BIC) demonstrated optimal overall performance in model selection.
• The efficacy of other selection techniques was contingent on the time-series length.
• Model selection performance varied significantly across different nonstationary process types.

Conclusions:

• Data-driven model selection alone is insufficient for nonstationary time-series.
• Integrating theory-driven insights with data-driven approaches enhances model selection accuracy.
• A hybrid approach combining qualitative understanding and quantitative methods is recommended for nonstationary time-series analysis.