An Item Response Model for True-False Exams Based on Signal Detection Theory

Area of Science:

• Educational Measurement
• Psychometrics
• Cognitive Psychology

Background:

• True-false exams are common assessment tools.
• Traditional models like Item Response Theory (IRT) do not fully capture the cognitive processes involved.
• Signal Detection Theory (SDT) offers a framework for understanding decision-making under uncertainty.

Purpose of the Study:

• To conceptualize true-false exams as a signal detection task.
• To develop an SDT-based model for analyzing true-false test performance.
• To compare the SDT model with existing Item Response Theory (IRT) and Grade of Membership (GOM) models.

Main Methods:

• Framing the true-false exam as a signal detection task.
• Utilizing SDT principles to model examinee response processes.
• Comparing the proposed SDT model with IRT and GOM models using an algebra exam dataset.

Main Results:

• The SDT model explicitly incorporates examinees' perceptions and decision thresholds.
• SDT offers a distinct perspective on item characteristics like difficulty and guessing, viewing them as manifestations of item bias.
• The application to an algebra exam demonstrates the model's utility and provides comparative insights.

Conclusions:

• Signal Detection Theory provides a valuable alternative framework for analyzing true-false exams.
• SDT's emphasis on perceptual and decision components offers a more nuanced understanding of examinee responses.
• The SDT approach can potentially refine psychometric analyses by accounting for subjective examinee perceptions.