Estimating correlations across tasks in experimental psychology

Understanding how people covary in performance across experimental tasks is central to individual-difference psychology. The classic Pearson correlation has two strengths: (1) it is invariant to the scale of measurement, and (2) it is invariant to including additional variables in the analysis. However, it is susceptible to attenuation from measurement noise. Bayesian hierarchical models address this issue by modeling measurement error directly. Resulting estimates, however, depend on prior specifications and are not invariant to scale or variable inclusion. We compare three common priors-inverse Wishart (IW), scaled inverse Wishart (SIW), and LKJ-to assess robustness to prior assumptions in hierarchical settings. Our main tools are visualizing the priors and evaluating their effects on posterior estimates through simulation. When prior settings match ground truth, all priors recover true correlations accurately in low-dimensional settings. When prior variance is misspecified, the IW shows strong bias: low-variance priors inflate correlations, and high-variance priors deflate them. The SIW shows the same pattern but less severely, while the LKJ remains largely unaffected by scale misspecification. When more variables are added, the IW is most stable, whereas the SIW and LKJ show slight shrinkage toward lower correlations. The main drawback of the LKJ is computational speed-models with it can take orders of magnitude longer than those using IW or SIW. Overall, the LKJ provides the most accurate estimates, while the SIW offers a practical compromise for large-scale models where computational speed is crucial.