Hierarchical Bayesian Regression for experimental psychology: a case study of cognitive control

Arising from the so-called 'replication crisis' in the experimental psychology literature, there has been a growing call to reassess whether specific analytic practices might enhance the accuracy and precision of reported findings. This issue is explored here, through a case study examination of two previously collected datasets from the Dual Mechanisms of Cognitive Control (DMCC) task battery. This case study highlights the unique advantages afforded by Hierarchical Bayesian Regression (HBR) models as a potentially more rigorous analytic approach to statistical inference. In the DMCC datasets, two sets of HBR models are presented, with the estimates of the former used as priors for the latter. In addition to systematically generating cumulative posterior distributions for all effects of theoretical interest, we further illustrate how our particular application of HBR models provides novel insights regarding specific indicators of proactive/reactive control in each of the four DMCC tasks, by: (1) estimating the consistency of effects across datasets; (2) estimating the relative strength of null effects; (3) accurately modeling the specific properties of response time distributions; and (4) appropriately modeling accuracy patterns at the trial level.