Selecting cut scores with a composite of item types: the construct mapping procedure

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• Setting accurate cut scores is crucial for test interpretation.
• Existing methods for cut score setting lack a unified conceptual framework.
• Item response theory (IRT) offers a robust framework for test analysis.

Purpose of the Study:

• To introduce a new, coherent method for setting test cut scores.
• To integrate item response modeling and the Wright Map into a practical procedure.
• To evaluate the new method's effectiveness and user reception.

Main Methods:

• Developed a novel cut score setting method based on item response modeling.
• Utilized the Wright Map to visualize item and threshold difficulties.
• Employed computer software to calculate response probabilities for potential cut scores.
• Conducted a validation study on the Golden State Examination in Chemistry.

Main Results:

• Investigated classification agreement between two groups using the new method.
• Assessed committee members' reactions to the procedure and software.
• Identified practical lessons learned from the implementation process.

Conclusions:

• The proposed method provides a structured approach to cut score setting.
• Integration of IRT and the Wright Map offers a powerful tool for educational measurement.
• The study highlights the importance of user feedback in refining psychometric procedures.