Optimal Reassembly of Shadow Tests in CAT

Area of Science:

• Educational Measurement
• Computer Science

Background:

• Large-scale online testing programs face concurrency challenges impacting computer-adaptive test (CAT) delivery.
• The shadow-test approach, while effective, can be computationally intensive due to frequent test reassemblies.

Purpose of the Study:

• To reduce the number of shadow test reassemblies in operational CAT programs.
• To maintain measurement quality while increasing concurrency for large-scale testing.

Main Methods:

• Explored strategies for reducing shadow test reassemblies, including fixed intervals and minimum change in ability estimates.
• Investigated a hybrid approach combining fixed intervals and ability estimate changes.
• Evaluated strategies for their impact on reassembly frequency and measurement accuracy.

Main Results:

• Substantial reductions in shadow test reassemblies were achieved across tested strategies.
• No degradation in measurement accuracy was observed with the implemented strategies.
• Strategies prevented unnecessary reassemblies caused by early-stage estimation noise.

Conclusions:

• The explored strategies enhance the efficiency and practicality of the shadow-test approach for large-scale CAT.
• Minimizing computational load associated with mixed-integer programming in shadow testing is feasible.
• Optimized reassembly strategies improve the robustness of online adaptive testing programs.