The impact of data collection design, linking method, and sample size on vertical scaling using the Rasch model

The Rasch model-based vertical scaling was evaluated by simulation study with respect to recovery of item parameter, linking constant, population mean (grade-to-grade growth), population standard deviation (grade-to-grade variability), and separation of grade distributions by effect size. The simulated vertical scale had five different grades with five different test levels. Controlled factors were data collection design, linking methods, and sample size. For item parameter, linking constant, and population mean, counter-balanced single group (CBSG) with mean/mean (or fixed item) method and concurrent calibration performed best. The population standard deviation recovery, as sample size increases, did not show systematic improvement across different data collection and linking methods. For the separation of grade distributions, CBSG with mean/mean (or fixed item) methods performed best. The average absolute differences from the true parameters were less than 0.1 in logit across different linking methods. In general the differences between different linking methods were less than those between different sample sizes.