An Investigation of Exposure Control Methods With Variable-Length CAT Using the Partial Credit Model

Area of Science:

• Psychometrics
• Educational Measurement
• Computerized Adaptive Testing (CAT)

Background:

• Limited research exists on item exposure control in computerized adaptive testing (CAT) with polytomous items and variable-length tests.
• Understanding the impact of different exposure control procedures and stopping rules is crucial for optimizing CAT efficiency and precision.

Purpose of the Study:

• To investigate the effects of various item exposure control procedures within CAT using variable-length stopping rules and the partial credit model.
• To compare the performance of randomesque (group of 3 and 6) and progressive-restricted standard error (PR-SE) exposure control methods.
• To evaluate the efficacy of two variable-length stopping rules: standard error (SE) and predicted standard error reduction (PSER).

Main Methods:

• A simulation study was conducted using the partial credit model.
• Four exposure control techniques were examined: randomesque (group of 3), randomesque (group of 6), PR-SE, and no control.
• Two variable-length stopping rules (SE and PSER) were implemented with item pools of 43, 86, and 172 items.

Main Results:

• The PSER stopping rule administered fewer items on average than the SE rule while maintaining comparable measurement precision across various conditions.
• The PR-SE exposure control method demonstrated superior performance over randomesque methods, reducing test overlap and controlling item exposure rates.
• PR-SE led to better pool utilization with minimal increases in administered items and nonconvergent cases.

Conclusions:

• The PSER stopping rule is effective for reducing test length in CAT without compromising measurement precision.
• The PR-SE exposure control procedure offers significant advantages in managing item overlap and exposure rates, enhancing overall CAT efficiency.
• These findings provide valuable insights for developing more effective CAT systems, particularly for assessments using polytomous items.