Comparing methods for assessing a difference in correlations with dependent groups, measurement error, nonnormality,...

Area of Science:

• Psychometrics
• Statistical Modeling
• Correlational Analysis

Background:

• Fisher's z transformation is common for testing differences in correlations but can be inaccurate with measurement error or non-normal data.
• Existing methods often use raw scores, neglecting potential issues like measurement error.
• Structural Equation Modeling (SEM) provides a framework to address these limitations.

Purpose of the Study:

• To compare various statistical methods for estimating and testing differences in correlations (ρdiff) between two variables.
• To evaluate the performance of these methods under conditions of measurement error, non-normality, and missing data.
• To provide recommendations for appropriate statistical approaches based on study findings.

Main Methods:

• Utilized Structural Equation Modeling (SEM) with latent-variable modeling to account for measurement error.
• Employed maximum likelihood estimation (ML) with sandwich-type standard errors, bootstrap confidence intervals, and Bayesian credible intervals for non-normal data.
• Assessed SEM frameworks under conditions of complete and missing data, including mean- and variance-adjusted likelihood ratio tests and bootstrapping model fit tests.

Main Results:

• SEM effectively handles measurement error and non-normality when testing differences in correlations.
• Different SEM approaches (ML with various intervals, Bayesian) show varying performance depending on data characteristics.
• The study identified robust methods for testing ρdiff across different data conditions, including missing data scenarios.

Conclusions:

• SEM is a recommended approach for testing differences in correlations, particularly when measurement error or non-normality is present.
• The choice of specific SEM techniques (e.g., bootstrap vs. Bayesian intervals) depends on the data's distributional properties and completeness.
• Findings guide researchers in selecting appropriate statistical methods for complex correlational analyses.