A new sample-size planning approach for person-specific VAR(1) studies: Predictive accuracy analysis.

Area of Science:

• Psychology
• Statistics
• Data Science

Background:

• N=1 studies are increasingly used to examine short-term dynamic processes within individuals.
• Vector Autoregression (VAR(1)) models are commonly applied to analyze data from N=1 studies.
• Determining the necessary number of measurement occasions for sample-size planning is critical.

Purpose of the Study:

• To introduce a novel simulation-based sample-size planning method, Predictive Accuracy Analysis (PAA).
• To address the limitations of traditional power analysis in N=1 studies by incorporating model overfitting.
• To provide a new metric for assessing out-of-sample predictive accuracy in multivariate time-series data.

Main Methods:

• Developed a simulation-based approach (PAA) for sample-size planning.
• Proposed a novel multivariate predictive accuracy metric.
• Utilized a Shiny app to facilitate the PAA method.
• Compared PAA with traditional power analysis using simulated and real data.

Main Results:

• Predictive accuracy analysis offers insights into potential model overfitting, a factor not addressed by power analysis.
• PAA generally recommends smaller sample sizes compared to power analysis.
• VAR(1) model parameters influence both power and predictive accuracy-based sample-size recommendations.

Conclusions:

• Predictive accuracy analysis (PAA) provides a valuable complement to power analysis for sample-size planning in N=1 studies.
• PAA enhances the understanding of model generalizability and potential overfitting.
• The proposed PAA method and associated app can aid researchers in optimizing sample size for N=1 dynamic process research.