Fused SDT/IRT Models for Mixed-Format Exams

Area of Science:

• Psychometrics
• Educational Measurement
• Cognitive Psychology

Background:

• Mixed-format exams combine multiple-choice (MC) and open-ended (OE) items.
• Existing models like signal detection theory (SDT) for MC and item response theory (IRT) for OE items operate independently.
• A unified framework is needed to understand the psychological processes underlying performance across different item formats.

Purpose of the Study:

• To propose a unified psychological framework for analyzing mixed-format exams.
• To integrate SDT and IRT models through the concept of examinee knowledge.
• To provide a method for simultaneously analyzing MC and OE items.

Main Methods:

• A choice model based on signal detection theory (SDT) for MC items.
• An item response theory (IRT) model for OE items.
• A general model fusing SDT and a generalized sequential logit model for OE items, linked by the probability of knowing.

Main Results:

• SDT and IRT models share a common conceptualization of latent 'know/don't know' states.
• The proposed fused model allows simultaneous fitting of SDT and IRT.
• The approach was illustrated using data from large-scale international exams.

Conclusions:

• The fused model offers a comprehensive approach to mixed-format test analysis.
• Simultaneous fitting allows examination of psychological process differences and covariate effects.
• This framework provides potential estimation benefits and deeper insights into examinee cognition.