How much data do we need to estimate computational models of decision-making? The COMPASS toolbox

How much data are needed to obtain useful parameter estimations from a computational model? The standard approach to address this question is to carry out a goodness-of-recovery study. Here, the correlation between individual-participant true and estimated parameter values determines when a sample size is large enough. However, depending on one's research question, this approach may be suboptimal, potentially leading to sample sizes that are either too small (underpowered) or too large (overcostly or unfeasible). In this paper, we formulate a generalized concept of statistical power and use this to propose a novel approach toward determining how much data is needed to obtain useful parameter estimates from a computational model. We describe a Python-based toolbox (COMPASS) that allows one to determine how many participants are needed to fit one specific computational model, namely the Rescorla-Wagner model of learning and decision-making. Simulations revealed that a high number of trials per person (more than the number of persons) are a prerequisite for high-powered studies in this particular setting.