Effects of violating local independence on IRT parameter estimation for the Binomial Trials Model

The appropriateness of the Binomial Trials Model for test data that consist of multiple attempts of the same item needs to be determined because the presence of learning or fatigue effects may violate the model's assumption of local independence. The purpose of this study was to determine what effect the severity of the violation of local independence (VLI), coupled with different sample size (SS), test length (TL), and test difficulty (TD) had on the estimation of the model difficulty parameter, b, using computer simulation techniques. Each of the following conditions was replicated 100 times under a completely crossed design: SS (100, 200, 500, 2,000); TL (5, 10, 20, 25 attempts); TD (-1.2, 0.0, 1.2); and VLI (from no violation to complete violation). Examinee ability or latent trait was pseudorandomly drawn from a standard normal distribution, and the b-parameter was estimated using a maximum likelihood procedure on generated test scores. Regardless of SS, TL, and TD, the b-parameter tended to be overestimated for situations in which the VLI condition simulated fatigue and underestimated when the VLI condition simulated late-test learning or practice effect. The findings suggest that violations of local independence, at least as simulated in this study, could seriously bias the difficulty parameter estimates if all examinees tested exhibited the dependency.