Selecting cut scores with a composite of item types: the construct mapping procedure

In this paper, we describe a new method we have developed for setting cut scores between levels of a test. We outline the wide variety of potential methods that have been used for such a process, and emphasize the need for a coherent conceptual framework under which the variety of methods could be understood. We then describe our particular method, based on an item response modeling framework, which uses the Wright Map, a graphical model of item and threshold difficulties, and a piece of computer software that provides probabilities of various responses for scores under consideration as cut scores. Finally, we describe a study we conducted for the Golden State Examination in Chemistry, in which we investigate the classification agreement for two groups using the method, and also investigate the reactions of the committee members to the procedure and the software, and the lessons we learned from this process.