Evaluating Testing, Profile Likelihood Confidence Interval Estimation, and Model Comparisons for Item Covariate...

Area of Science:

• Psychometrics
• Statistical Modeling
• Educational Measurement

Background:

• The linear logistic test model (LLTM) is used to study item covariate effects on difficulty.
• The LLTM was extended to the LLTM-R by incorporating random item residuals.
• Investigating statistical inference for these models is crucial for accurate analysis.

Purpose of the Study:

• To compare statistical inference methods for the LLTM and LLTM-R.
• To evaluate Type I error rates and statistical power via Monte Carlo simulations.
• To determine the most reliable model and testing approach for item covariate analysis.

Main Methods:

• Monte Carlo simulations were employed to compare Type I error rates and power.
• Statistical inference methods including the likelihood ratio test (LRT), paired-sample t test, Wald z test, and information criteria were assessed.
• The performance of the LLTM and LLTM-R was evaluated under varying conditions of residual variance and sample sizes.

Main Results:

• The likelihood ratio test (LRT) is recommended over other methods due to its simplicity and reliability.
• The LLTM-R is identified as the better general model, as inferences from the LLTM can be biased when LLTM-R is the true model.
• Adequate power and acceptable Type I error rates require a large number of items (80) when residual variance is present.

Conclusions:

• The LLTM-R provides more accurate inferences than the LLTM, especially when unexplained item variance exists.
• The LRT is the recommended statistical test for inference in both LLTM and LLTM-R.
• Sufficient sample size and number of items are critical for reliable results, particularly in the presence of random item residuals.