Performance of the S ² Statistic for the Multidimensional Graded Response Model.

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• The $$ statistic is widely used for item fit assessment in item response theory (IRT).
• Its performance within the multidimensional graded response model (MGRM) remains under-investigated.
• Understanding $$ behavior is crucial for accurate psychometric analysis.

Purpose of the Study:

• To systematically evaluate the performance of the $$ item fit index under the MGRM.
• To investigate $$ under scenarios of overall model misspecification and localized item misfit.
• To assess the influence of sample size and test length on $$ accuracy.

Main Methods:

• Monte Carlo simulation studies were employed to generate data under various MGRM conditions.
• The study examined two primary misfit scenarios: global model misspecification and subset item misfit.
• Performance was assessed based on true positive rates (TPRs) and false positive rates (FPRs) of the $$ index.

Main Results:

• The $$ index demonstrated inflated false positive rates (FPRs) for polytomous items under the MGRM, especially with small sample sizes and long tests.
• $$ showed good performance in detecting overall model misfit and item misfit when the ordinality assumption was violated.
• Under model misspecification or violations of the homogeneous discrimination assumption, inflated FPRs often increased with TPRs, particularly in small samples with long tests.

Conclusions:

• The use of the $$ index for item fit in the MGRM requires careful consideration due to potentially high false positive rates.
• While $$ can detect certain types of misfit, its reliability is compromised under specific conditions of model misspecification and sample/test characteristics.
• Researchers should exercise caution when interpreting $$ results for polytomous items within the MGRM framework.