Estimating person parameters via item response model and simple sum score in small samples with few polytomous items:...

Area of Science:

• Psychometrics
• Statistical Modeling

Background:

• Item Response Theory (IRT) is increasingly utilized for item analysis.
• Precise parameter estimates in IRT typically require numerous items and large sample sizes.

Purpose of the Study:

• To investigate the performance of Item Response Theory (IRT) models versus simple sum scores under various data conditions.
• To determine optimal conditions for utilizing IRT models for accurate ability estimation.

Main Methods:

• A simulation study manipulated sample size (40-900), number of items (4-32), item distribution (symmetric vs. skew), item loadings (homogeneous vs. heterogeneous, low vs. high), correlated errors, and number of response categories (4 vs. 5).
• Missing data (10%) was introduced.
• Ability estimates from IRT and simple sum scores were compared.

Main Results:

• IRT models outperformed sum scores with many items, skewed distributions, and heterogeneous/high item loadings.
• Sum scores were superior with few items, symmetric distributions, and homogeneous/low item loadings.
• Convergence rates were low in small samples, and correlated errors negatively impacted both methods.

Conclusions:

• IRT models are recommended for skewed item distributions and heterogeneous item loadings.
• Sufficient sample size or item count is crucial for IRT accuracy; with limited data, sum scores may be more effective.
• Correlated errors pose a challenge for both IRT and sum score accuracy.