Random Item Response Data Generation Using a Limited-Information Approach: Applications to Assessing Model Complexity

Area of Science:

• Psychometrics
• Statistical Modeling
• Educational Measurement

Background:

• Fitting propensity (FP) analysis quantifies model complexity in item response theory (IRT).
• Traditional full-information approaches face computational challenges in sampling response patterns.
• Limited-information (LI) methods offer a viable alternative for IRT model evaluation.

Purpose of the Study:

• To develop an efficient algorithm for sampling item response patterns in IRT.
• To enable the evaluation of fitting propensity (FP) using a limited-information (LI) approach.
• To compare the configural complexity of different IRT models.

Main Methods:

• Developed the Sequential Importance Sampling algorithm to Quickly and Uniformly Obtain Contingency tables (SISQUOC).
• Employed a limited-information (LI) approach, generating data from lower-order margins.
• Utilized an iterative proportional fitting procedure to reconstruct joint probabilities for FP evaluation.

Main Results:

• The SISQUOC algorithm effectively generates large, uniformly random datasets for IRT.
• The LI approach simplifies data generation for both dichotomous and polytomous items.
• Analysis of graded response and generalized partial credit models indicates similar configural complexity.

Conclusions:

• The proposed LI approach and SISQUOC algorithm overcome computational barriers in IRT FP analysis.
• This method facilitates robust model complexity assessment in item response theory.
• The study provides insights into the configural complexity of common IRT models.