Are alternative variables in a set differently associated with a target variable? Statistical tests and practical...

Area of Science:

• Statistics
• Psychometrics
• Behavioral Sciences

Background:

• Bivariate correlation analysis is common in research.
• Comparing dependent correlations often relies on informal methods or individual significance tests.
• Existing statistical tests for differences between dependent correlations with overlapping variables are not always well-understood or applied correctly.

Purpose of the Study:

• To evaluate the accuracy, power, and robustness of ten statistical tests for comparing two dependent correlations with overlapping variables.
• To identify reliable and appropriate statistical methods for this specific comparison scenario.
• To provide practical guidance for researchers selecting a test.

Main Methods:

• Simulation methods were employed to assess test performance.
• Empirical Type I error rates (accuracy) were calculated under various conditions.
• Statistical power and robustness against non-normality were also evaluated.
• Ten distinct statistical tests for dependent correlations were compared.

Main Results:

• Five of the ten tested methods demonstrated unacceptable empirical Type I error rates, deviating significantly from the nominal alpha level.
• The remaining five tests were deemed acceptable, showing similar performance across accuracy, power, and robustness criteria.
• No single test demonstrated robustness across all explored forms of non-normality.
• Performance varied depending on the parameter space and distributional assumptions.

Conclusions:

• Researchers should exercise caution when choosing statistical tests to compare dependent correlations with overlapping variables.
• Five of the evaluated tests are not recommended due to poor accuracy.
• The remaining five tests offer viable options, but their robustness limitations should be considered.
• Practical recommendations are provided to aid in selecting the most appropriate test based on study specifics.