A Systematic Study into the Factors that Affect the Predictive Accuracy of Multilevel VAR(1) Models

Area of Science:

• Psychological research methods
• Quantitative psychology
• Longitudinal data analysis

Background:

• Multilevel vector autoregression (VAR(1)) models are increasingly used for analyzing intensive longitudinal data in psychology.
• These models capture within-individual dynamics by estimating auto- and cross-regressive relationships while accounting for individual differences.
• Assessing the generalizability of model estimates to unseen data, via cross-validation, is crucial for model quality.

Purpose of the Study:

• To systematically investigate factors influencing the predictive accuracy of multilevel VAR(1) models.
• To extend previous work on cross-validation for assessing the generalizability of these models.
• To provide practical guidance for researchers using multilevel VAR(1) models with complex datasets.

Main Methods:

• Conducted three simulation studies to evaluate predictive accuracy under varying conditions.
• Manipulated key factors: number of measurement occasions, number of persons, number of variables, contemporaneous collinearity, and distributional shape of individual differences.
• Employed cross-validation techniques to assess how well models generalize to unseen data.

Main Results:

• Pooling information across individuals and employing multilevel techniques effectively prevent overfitting.
• Reducing the variable space is beneficial when strong contemporaneous correlations are expected among variables.
• Multilevel VAR(1) models incorporating random effects demonstrate superior predictive performance compared to person-specific models when individuals share similar dynamics.

Conclusions:

• Multilevel VAR(1) models are robust and generalize well, especially when leveraging information across individuals.
• Model complexity and variable interdependencies significantly impact predictive accuracy, guiding model selection.
• The findings support the use of multilevel VAR(1) models with random effects for capturing population-level insights from intensive longitudinal data.