Assessing Dimensionality of IRT Models Using Traditional and Revised Parallel Analyses.

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• Accurate dimensionality assessment is crucial for applying item response theory (IRT) models.
• Traditional and revised parallel analyses exist within factor analysis but lack systematic evaluation in IRT.

Purpose of the Study:

• To evaluate the accuracy of traditional and revised parallel analyses for dimensionality determination within the IRT framework.
• To compare the performance of these methods under various data generation conditions.

Main Methods:

• Conducted simulation studies manipulating six factors: sample size, test length, model type, number of dimensions, inter-dimensional correlations, and item discrimination.
• Assessed the performance of traditional parallel analysis (PCA with tetrachoric correlation) and other methods.

Main Results:

• Traditional parallel analysis (PCA with tetrachoric correlation) performed best for unidimensional IRT models across all conditions.
• This method also yielded the highest accuracy for multidimensional IRT models, with exceptions for high inter-dimensional correlations (0.8) or low item discrimination.
• Some factor combinations, like three-dimensional 3PL models with low discrimination and high correlation, resulted in poor performance for all tested methods.

Conclusions:

• Traditional parallel analysis using PCA and tetrachoric correlation is a reliable method for dimensionality assessment in IRT.
• Care must be taken when interpreting results under specific conditions of high dimensionality and item characteristics.