Examining the tradeoffs of exposure control and collateral information with multidimensional forced-choice...

Area of Science:

• Psychometrics
• Educational Measurement
• Psychological Testing

Background:

• Multidimensional forced-choice (MFC) testing is an alternative to single-statement (SS) Likert-type measures to reduce response biases.
• Computerized adaptive testing (CAT) enhances testing efficiency by selecting informative items.
• Existing MFC CAT methods lead to item overuse, impacting test security and costs.

Purpose of the Study:

• Develop an exposure control method for MFC CAT using the multi-unidimensional pairwise preference model (MUPP).
• Investigate if collateral information can offset potential losses in accuracy and precision caused by exposure control.
• Evaluate the effectiveness of the exposure control method and collateral information in MFC CAT.

Main Methods:

• Developed an exposure control method for MFC CAT based on the MUPP.
• Conducted a Monte Carlo simulation study.
• Manipulated test length and correlation between dimensions.

Main Results:

• The developed exposure control method improved test security and cost-effectiveness.
• Incorporating collateral information helped offset losses in measurement accuracy and precision.
• The MFC CAT algorithm maintained measurement accuracy and precision.

Conclusions:

• The new exposure control method effectively addresses item overuse in MFC CAT.
• Collateral information can be integrated to preserve measurement quality.
• The research provides an MFC CAT algorithm that balances efficiency, security, and accuracy.