Performance of the likelihood ratio difference (G2 Diff) test for detecting unidimensionality in applications of the...

Area of Science:

• Psychometrics
• Statistical modeling
• Item response theory

Background:

• Previous research explored the likelihood ratio difference test in item response models concerning sample size, model misspecification, test length, ability distribution, and generating models.
• This study extends prior investigations to assess dimensionality using the multidimensional random coefficients multinomial logit model (MRCMLM).

Purpose of the Study:

• To evaluate the suitability of the likelihood ratio difference test for assessing dimensionality within the MRCMLM framework.
• To investigate the influence of sample size and test length on the Type I error rates of the likelihood ratio difference test in MRCMLM applications.

Main Methods:

• Simulated data were generated and analyzed using logistic regression.
• The study focused on the impact of varying sample sizes, test lengths, and levels of data misfit on the likelihood ratio difference test's performance.

Main Results:

• Sample size and test length significantly affect the likelihood ratio difference test's ability to identify unidimensionality.
• Smaller sample sizes and shorter test lengths resulted in reduced Type I error rates.
• Higher levels of simulated misfit led to fewer incorrect decisions compared to data with minimal misfit.

Conclusions:

• The likelihood ratio difference test demonstrates a substantial Type I error rate across all simulated conditions.
• The likelihood ratio difference test is not recommended for evaluating dimensionality in the context of the MRCMLM.
• Further research may be needed to identify suitable methods for dimensionality assessment in MRCMLM applications.