Effects of violating local independence on IRT parameter estimation for the Binomial Trials Model.

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• The Binomial Trials Model assumes local independence, which may be violated by learning or fatigue effects in repeated item attempts.
• Accurate estimation of item difficulty parameters is crucial for reliable test construction.

Purpose of the Study:

• To investigate the impact of violating local independence on the Binomial Trials Model's difficulty parameter (b) estimation.
• To examine how sample size (SS), test length (TL), and test difficulty (TD) interact with the severity of violation of local independence (VLI).

Main Methods:

• Computer simulation techniques were employed under a completely crossed design.
• Replications (n=100) were conducted across varying SS (100-2,000), TL (5-25 attempts), TD (-1.2 to 1.2), and VLI levels.
• Examinee ability was drawn from a standard normal distribution, and the b-parameter was estimated using maximum likelihood.

Main Results:

• The difficulty parameter (b) was overestimated when VLI simulated fatigue effects.
• The difficulty parameter (b) was underestimated when VLI simulated late-test learning or practice effects.
• These biases occurred regardless of sample size, test length, and test difficulty.

Conclusions:

• Violations of local independence can significantly bias difficulty parameter estimates in the Binomial Trials Model.
• The direction and magnitude of bias depend on the nature of the dependency (fatigue vs. learning).
• Careful consideration of local independence is necessary when applying the Binomial Trials Model to data with repeated item attempts.