Parameters, Predictions, and Evidence in Computational Modeling: A Statistical View Informed by ACT-R

Model validation in computational cognitive psychology often relies on methods drawn from the testing of theories in experimental physics. However, applications of these methods to computational models in typical cognitive experiments can hide multiple, plausible sources of variation arising from human participants and from stochastic cognitive theories, encouraging a "model fixed, data variable" paradigm that makes it difficult to interpret model predictions and to account for individual differences. This article proposes a likelihood-based, "data fixed, model variable" paradigm in which models are treated as stochastic processes in experiments with participant-to-participant variation that can be applied to a broad range of mechanistic cognitive architectures. This article discusses the implementation and implications of this view in model validation, with a concrete focus on a simple class of ACT-R models of cognition. This article is not intended as a recipe for broad application of these preliminary, proof-of-concept methods, but as a framework for communication between statisticians searching for interesting problems in the cognitive modeling sphere, and cognitive modelers interested in generalizing from deterministic to stochastic model validation, in the face of random variation in human experimental data.