Impact of Not Addressing Partially Cross-Classified Multilevel Structure in Testing Measurement Invariance: A Monte
Myung H Im1, Eun S Kim2, Oi-Man Kwok3
1American Institutes of ResearchWashington, DC, USA; Department of Educational Psychology, Texas A&M UniversityCollege Station, TX, USA.
Frontiers in Psychology
|April 6, 2016
Summary
Researchers often face cross-classified data in education. Conventional multilevel modeling struggles with measurement invariance testing, while cross-classified MIMIC models offer a viable solution for accurate analysis.
Area of Science:
- Educational research
- Psychometrics
- Statistical modeling
Background:
- Multilevel data with cross-classified structures are common in educational research.
- Researchers often use suboptimal methods for analyzing such data, particularly for measurement invariance testing.
- Existing statistical software limitations hinder the adoption of appropriate cross-classified models.
Purpose of the Study:
- To investigate the performance of measurement invariance testing in cross-classified multilevel data.
- To compare the conventional multilevel confirmatory factor analysis (MCFA) with cross-classified MIMIC models.
- To evaluate the impact of ignoring a crossed factor in MCFA and the adequacy of cross-classified MIMIC models.
Main Methods:
- Two Monte Carlo studies were conducted.
- Examined the impact of ignoring a crossed factor using conventional MCFA.
- Assessed the performance of cross-classified MIMIC models for measurement invariance.
- Varied intraclass correlation (ICC) and magnitude of non-invariance.
Main Results:
- Conventional MCFA showed very low statistical power to detect non-invariance.
- Underestimated factor loading differences and ICC in MCFA due to ignoring the crossed factor.
- Cross-classified MIMIC models demonstrated acceptable performance with cross-classified data.
Conclusions:
- Conventional MCFA can lead to incorrect statistical inferences when applied to cross-classified data.
- Treating cross-classified data as hierarchical in MCFA is inappropriate for measurement invariance testing.
- Cross-classified MIMIC models are recommended for accurate measurement invariance analysis in educational research with cross-classified data.
Related Concept Videos
00:55Ordinal Level of Measurement
36.8K
The way a set of data is measured is called its level of measurement. Correct statistical procedures depend on a researcher being familiar with levels of measurement. For analysis, data are classified into four levels of measurement—nominal, ordinal, interval, and ratio.
Data measured using an ordinal scale are similar to nominal scale data, but there is one major difference. The ordinal scale data can be ordered. An example of ordinal scale data is a list of the top five national parks...
Data measured using an ordinal scale are similar to nominal scale data, but there is one major difference. The ordinal scale data can be ordered. An example of ordinal scale data is a list of the top five national parks...
36.8K
01:15One-Way ANOVA: Equal Sample Sizes
4.4K
One-Way ANOVA can be performed on three or more samples with equal or unequal sample sizes. When one-way ANOVA is performed on two datasets with samples of equal sizes, it can be easily observed that the computed F statistic is highly sensitive to the sample mean.
Different sample means can result in different values for the variance estimate: variance between samples. This is because the variance between samples is calculated as the product of the sample size and the variance between the...
Different sample means can result in different values for the variance estimate: variance between samples. This is because the variance between samples is calculated as the product of the sample size and the variance between the...
4.4K
01:15One-Way ANOVA: Unequal Sample Sizes
6.9K
One-way ANOVA can be performed on three or more samples of unequal sizes. However, calculations get complicated when sample sizes are not always the same. So, while performing ANOVA with unequal samples size, the following equation is used:
6.9K
00:56Nominal Level of Measurement
41.7K
The way a set of data is measured is called its level of measurement. Correct statistical procedures depend on a researcher being familiar with levels of measurement. Not every statistical operation can be used with every set of data. For analysis, data are classified into four levels of measurement—nominal, ordinal, interval, and ratio.
The data that cannot be measured but can be grouped into categories fall under the nominal level of measurement. Data that is measured using a nominal...
The data that cannot be measured but can be grouped into categories fall under the nominal level of measurement. Data that is measured using a nominal...
41.7K
01:25Strategies for Assessing and Addressing Confounding
528
Confounding is a critical issue in epidemiological studies, often leading to misleading conclusions about associations between exposures and outcomes. It occurs when the relationship between the exposure and the outcome is mixed with the effects of other factors that influence the outcome. Given that, addressing confounding is of high importance for drawing accurate inferences in research.
Confounding can be addressed at both the design phase of a study and through analytical methods after data...
Confounding can be addressed at both the design phase of a study and through analytical methods after data...
528
01:18One-Way ANOVA
14.4K
One-way ANOVA analyzes more than three samples categorized by one factor. For example, it can compare the average mileage of sports bikes. Here, the data is categorized by one factor - the company. However, one-way ANOVA cannot be used to simultaneously compare the sample mean of three or more samples categorized by two factors. An example of two factors would be sports bikes from different companies driven in different terrains, such as a desert or snowy landscape. Here, two-way ANOVA is used...
14.4K