An Approach to Scoring and Equating Tests With Binary Items: Piloting With Large-Scale Assessments.

Area of Science:

• Psychometrics
• Educational Measurement
• Quantitative Psychology

Background:

• Traditional test scoring methods can be complex and prone to errors in large-scale assessments.
• Item Response Theory (IRT) provides a framework for test calibration but can involve intricate equating procedures.

Purpose of the Study:

• To introduce and evaluate a new test scoring approach called delta scoring (D-scoring).
• To demonstrate the advantages of D-scoring over existing IRT-based methods for large-scale assessments.

Main Methods:

• D-scoring utilizes examinee response vectors weighted by expected item difficulties derived from IRT parameters.
• Expected item difficulties are calculated as analytic functions of item IRT parameters.
• The D-scale's interval properties are compared against the IRT logit scale.

Main Results:

• D-scores are independent of the specific sample of test-takers.
• The D-scale demonstrates superior intervalness compared to the IRT logit scale.
• Equating D-scales requires simple rescaling of item parameters, simplifying the process.

Conclusions:

• D-scoring offers a promising and efficient alternative for test scoring in large-scale educational and psychological assessments.
• The method simplifies equating procedures, reducing complexity and potential for error.
• D-scoring has the potential to complement existing IRT procedures effectively.