Multidimensional Test Assembly Using Mixed-Integer Linear Programming: An Application of Kullback-Leibler Information

Area of Science:

• Educational Measurement
• Psychometrics
• Computer Science (Optimization)

Background:

• Educational testing requires parallel test forms with no item overlap but similar statistical properties.
• Mixed-integer linear programming (MILP) is used for test assembly, with Fisher information (FI) as a common target in unidimensional cases.
• Multidimensional FI complicates its use as a target for linear test form assembly, and existing multidimensional computerized adaptive testing (MCAT) criteria are not directly applicable.

Purpose of the Study:

• To derive and evaluate new statistical targets for assembling multidimensional parallel test forms using MILP.
• To address the limitations of using multidimensional Fisher information (FI) in MILP for linear test assembly.
• To compare the effectiveness of FI-based and Kullback-Leibler (KL) divergence-based targets.

Main Methods:

• Derivation of novel statistical targets based on Fisher information (FI) and Kullback-Leibler (KL) divergence for MILP.
• Application of these targets in MILP models for assembling multidimensional parallel test forms.
• Evaluation using simulated and empirical item pools.

Main Results:

• The study successfully derived new statistical targets for multidimensional test assembly.
• Kullback-Leibler (KL) divergence-based targets demonstrated promising results in assembling parallel test forms.
• The proposed targets are suitable for MILP models in the context of linear test assembly.

Conclusions:

• New statistical targets, particularly those based on KL divergence, offer a viable solution for assembling multidimensional parallel test forms.
• The developed methods improve upon existing approaches for test assembly in educational measurement.
• Further research can explore the application and refinement of these targets in various testing contexts.