Performance of Four Multivariate Tests Under Variance-Covariance Heteroscedasticity

Area of Science:

• Multivariate statistical analysis
• Statistical hypothesis testing
• Covariance matrix analysis

Background:

• Accurate control of Type I error rates is crucial in multivariate hypothesis testing.
• Heterogeneous covariance matrices pose challenges to the performance of standard multivariate tests.

Purpose of the Study:

• To compare the Type I error rates (τ) of four multivariate tests under heterogeneous covariance matrices.
• To identify which tests maintain acceptable error rates in various simulated conditions.

Main Methods:

• Conducted 360 simulated experiments comparing Pillai-Bartlett trace, Johansen's test, James' first order test, and James' second order test.
• Evaluated test performance based on Type I error rates across different ratios of sample size to variables (N/p) and covariance matrix properties.

Main Results:

• Johansen's test and James' second order test demonstrated superior control of Type I error rates compared to Pillai-Bartlett trace and James' first order test.
• Johansen's test performed well when the N/p ratio was large and covariance matrices had smaller elements.
• James' second order test exhibited the best performance under extreme conditions, including small N/p, large covariance heterogeneity, and specific sample size/covariance element relationships.

Conclusions:

• Johansen's test and James' second order test are recommended for multivariate analyses involving heterogeneous covariance matrices.
• James' second order test offers enhanced robustness, particularly when dealing with challenging data characteristics and small sample sizes relative to the number of variables.