A Monte Carlo simulation study of sample size requirements for the Graded Response Model

Area of Science:

• Psychometrics
• Statistical Modeling

Background:

• The Graded Response Model (GRM) is widely used for analyzing ordinal data in psychometrics.
• Current sample size recommendations for GRM are often based on consensus, lacking empirical validation.
• The influence of the number of items (J) and response categories (K) on GRM parameter estimation accuracy needs further exploration.

Purpose of the Study:

• To investigate how sample size (n), number of items (J), and number of response categories (K) affect the estimation accuracy of latent trait and item discrimination parameters in the GRM.
• To provide empirical evidence to inform sample size recommendations for GRM applications.

Main Methods:

• A Monte Carlo simulation was employed to examine the effects of varying n (500-1500), J (5-50), and K (4-7) on GRM parameter estimation.
• Datasets were generated based on predefined distributions, and the GRM was fitted using the EM algorithm.
• Estimation accuracy was assessed using root mean squared error (RMSE) and Pearson's correlation coefficient.

Main Results:

• RMSE for the discrimination parameter (a) decreased with increased n and J; K had a negligible effect.
• RMSE for the latent trait parameter was primarily influenced by J, with minor impacts from n and K.
• High ordinal fidelity (Pearson's r > 0.98) was observed across all conditions, even with smaller sample sizes.

Conclusions:

• Sample size recommendations for GRM should be tailored to specific measurement goals.
• A sufficient number of items (J ≥ 30) can compensate for smaller sample sizes (n ≈ 500) for latent trait estimation.
• Precise estimation of item discrimination (a) necessitates larger sample sizes (n ≥ 1000), and increasing response categories (K) offers limited benefits for parameter recovery.