Estimating correlations across tasks in experimental psychology

Area of Science:

• Psychology
• Statistics
• Computational Modeling

Background:

• Individual-difference psychology relies on understanding performance covariation across tasks.
• Pearson correlation is scale-invariant but susceptible to measurement noise.
• Bayesian hierarchical models (BHMs) directly model measurement error for improved accuracy.

Purpose of the Study:

• Compare the robustness of three common priors (inverse Wishart [IW], scaled inverse Wishart [SIW], LKJ) in BHMs.
• Assess the impact of prior specification and variable inclusion on correlation estimates.
• Evaluate trade-offs between accuracy and computational speed.

Main Methods:

• Simulation studies to evaluate prior performance under various conditions.
• Visualization of prior distributions.
• Analysis of posterior estimates for correlation recovery and bias.

Main Results:

• All priors accurately recover correlations in low-dimensional settings when correctly specified.
• Inverse Wishart (IW) priors exhibit significant bias when variance is misspecified.
• LKJ priors are robust to scale misspecification and generally provide accurate estimates.
• SIW priors show similar bias patterns to IW but less severe.
• Adding variables stabilizes IW but causes shrinkage in SIW and LKJ.
• LKJ priors are computationally intensive compared to IW and SIW.

Conclusions:

• The LKJ prior is recommended for accurate correlation estimation in BHMs.
• The SIW prior offers a computationally efficient alternative for large-scale analyses.
• Prior choice significantly impacts BHM results, necessitating careful consideration.