Assessing Measurement Invariance Across Multiple Groups: When Is Fit Good Enough?

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• Large sample sizes are crucial for accurate parameter estimation and statistical power in complex research.
• Pooling extant datasets is a resource-efficient strategy to achieve large sample sizes.
• Measurement invariance (MI) modeling is essential for ensuring comparability of scores across different data sets.

Purpose of the Study:

• To present a novel, user-friendly method for addressing challenges in measurement invariance when combining independent datasets.
• To overcome issues of large sample sizes inflating statistical significance and varying measure combinations across datasets.
• To facilitate the pooling of diverse educational data while maintaining score comparability.

Main Methods:

• The proposed method combines generating random normal deviates for missing variables with assessing model fit using the root mean square error of approximation (RMSEA) "good enough principle".
• This approach hypothesizes that the difference between groups is small rather than strictly zero.
• The method was demonstrated by examining MI across eight independent datasets.

Main Results:

• The "good enough principle" approach demonstrated potential in handling measurement invariance across diverse datasets.
• Comparison with traditional MI methods showed differences in invariance decisions, particularly with large sample sizes.
• The user-friendly nature of the combined method was highlighted.

Conclusions:

• The presented method offers a practical solution for measurement invariance challenges in pooled datasets, especially in educational research.
• The "good enough principle" provides a more flexible criterion for assessing invariance with large samples.
• This approach enhances the feasibility of combining disparate educational datasets for robust analysis.