Bayesian regularization in multiple-indicators multiple-causes models.

Area of Science:

• Statistics
• Psychometrics
• Machine Learning

Background:

• Regularization methods are increasingly integrated into structural equation modeling (SEM) to enhance variable selection, model estimation, and prediction.
• Bayesian approaches offer a flexible framework for incorporating regularization priors into SEM.

Purpose of the Study:

• Compare various Bayesian regularization methods for analyzing covariate effects in multiple-indicators multiple-causes (MIMIC) models.
• Assess the impact of hyperparameter settings on penalty prior performance.
• Evaluate prediction accuracy using cross-validation.

Main Methods:

• A simulation study was conducted to compare Bayesian regularization methods including ridge, lasso, adaptive lasso, spike-and-slab prior (SSP), and horseshoe priors.
• These methods were applied to MIMIC models to estimate sparse structural coefficient matrices.
• Sensitivity analyses were performed on hyperparameter settings, and prediction accuracy was assessed via cross-validation.

Main Results:

• Penalty priors outperformed diffuse priors in small sample sizes and with collinear covariates.
• Global penalty priors (ridge, lasso) demonstrated higher convergence rates and power.
• Local and global penalty priors (horseshoe, SSP) provided more accurate parameter estimates and improved factor score prediction, leading to parsimonious models.

Conclusions:

• Bayesian regularization, particularly with horseshoe and SSP priors, is effective for variable selection and accurate parameter estimation in SEM.
• Penalty priors are crucial for handling challenging data conditions like small sample sizes and multicollinearity.
• These methods enhance prediction accuracy and model parsimony in MIMIC models.