Planning a Study for Testing the Rasch Model given Missing Values due to the use of Test-booklets

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• The Rasch model is widely used for achievement test calibration in psychology and education.
• Existing sample size determination methods often lack statistical rigor for data sampling.
• Previous simulation studies by Kubinger et al. (2009, 2011) focused on complete data scenarios.

Purpose of the Study:

• To investigate the applicability of a sample size determination approach for the Rasch model in the presence of missing data.
• To evaluate if the approach by Kubinger et al. (2009, 2011) remains valid when using multiple test booklets in large-scale assessments.
• To assess the impact of designed missing values on Type-I risk and test power.

Main Methods:

• A simulation study was conducted to analyze the three-way analysis of variance design with mixed classification.
• The study specifically addressed scenarios with missing values due to the use of multiple test booklets.
• The performance of the sample size approach was evaluated under conditions with dichotomous item response data.

Main Results:

• The simulation results indicate that the sample size determination approach is effective even with designed missing values.
• The use of test booklets did not significantly alter the actual Type-I risk or the power of the Rasch model test.
• Effectiveness was observed when examinees provided a comparable amount of information per item.

Conclusions:

• The validated approach for sample size determination is robust to missing data patterns typical in large-scale assessments.
• Researchers can confidently apply this method when using multiple test booklets, provided sufficient examinee information per item.
• It is recommended that researchers conduct their own simulations for specific scenarios due to the study's focused scope.